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EFFECTS  OF  CREW  COORDINATION  TRAINING  AND  EVALUATION  METHODS  ON 
AH-64  ATTACK  HELICOPTER  BATTALION  CREW  PERFORMANCE 

Introduction 

Dynamics  Research  Corporation  (DRC) ,  under  contract  to  the 
US  Army  Research  Institute  Aviation  Research  &  Development 
Activity  (USARIARDA) ,  developed,  evaluated,  and  validated  an 
exportable  Army  Crew  Coordination  training  and  evaluation 
package.  The  development  of  the  training  and  evaluation  systems 
and  the  results  of  the  US  Army  Aviation  Center  (USAAYNC) 
sponsored  crew  coordination  validation  testbed  conducted  by  air 
assault  battalion  crews  are  in  companion  reports  delivered  to 
USARIARDA  (Pawlik  et  al.,  1993;  Grubb  et  al.,  1993;  and  Simon  et 
al.,  1993).  The  following  report  describes  the  most  recent  phase 
of  this  research,  orchestrated  by  the  USARIARDA  and  USAAYNC  and 
technically  supported  by  DRC.  This  report  focuses  on  the  results 
of  providing  attack  helicopter  battalion  crews  with  crew 
coordination  training  in  preparation  for  follow-on  research  to 
assess  the  interaction  of  battle  rostering  and  crew  coordination 
training. 

Background 


Over  the  last  several  years,  USARIARDA  has  been  conducting  a 
program  of  training  research  that  responds  to  the  Army's  need  for 
better  crew  coordination  training.  This  program  of  research  has 
been  conducted  in  close  cooperation  with  the  USAAVNC  and  its 
effort  to  revise  its  training  standards  to  reflect  increased 
emphasis  on  crew-level  performance. 

Subsequent  to  the  initial  testing  of  the  training  and 
evaluation  system,  the  USAAVNC  formed  a  Working  Group  to 
incorporate  the  results  of  the  aircrew  coordination  research  into 
revisions  of  the  Aircrew  Training  Manuals  (ATMs)  for  all  Army 
aircraft.  Beginning  in  February  1992,  DRC  worked  closely  with 
the  USAAVNC  Working  Group  to  draft  training  and  evaluation 
methods  and  materials  for  a  crew  coordination  validation  testbed 
effort.  Drafts  of  the  Field  Exportable  Training  and  Evaluation 
packages  were  tested  and  validated  with  the  cooperation  of  the 
101st  Aviation  Brigade  (Air  Assault)  at  Fort  Campbell,  KY  from  3 
August  -  2  September  1992.  In  December  1992,  DRC  provided 
USARIARDA  and  USAAVNC  the  final  version  of  the  Aircrew 
Coordination  Exportable  Training  Package  to  implement  the 
recently  revised  ATMs. 

During  the  validation  testbed,  crews  from  the  5th  and  9th 
Air  Assault  Battalions  (UH-60)  at  Fort  Campbell,  KY  were  battle 
rostered,  that  is,  formed  into  pairs,  and  completed  four  missions 
in  the  flight  simulator  with  their  battle  rostered  crewmember. 

In  demonstrating  and  validating  the  new  field  exportable  program 
for  training  and  evaluating  crew  coordination  skills,  testbed 
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results  showed  that  the  crews  performed  their  missions  more 
effectively  and  safely  after  the  training  than  before  the 
training.  Validation  testbed  results  are  discussed  in  a  separate 
report  delivered  to  USARIARDA  (Simon  &  Grubb,  1993) .  Repeated 
flights  with  the  same  crewmember  surfaced  the  issue  of  the 
interactive  effect  of  battle  rostering  and  crew  coordination 
training  on  crew  performance.  The  USAAVNC  selected  the  229th 
Attack  Helicopter  Battalion  (AH-64)  Fort  Rucker,  AL  to  provide 
battle  rostered  crews  to  receive  crew  coordination  training  as 
the  first  phase  to  address  this  research  issue. 

Obj  ective 

To  determine  the  effects  of  crew  coordination  training  and 
evaluation  methods  on  AH-64  attack  helicopter  battalion  crew 
performance. 


Method 


Personnel 


y  V 


Five  standardization  instructor  pilots  (SIPs)  from  the 
USAAVNC s  Directorate  of  Evaluation  and  Standardization  (DES) 
received  an  abbreviated  version  of  the  Aircrew  Coordination 
Exportable  Training  Package,  called  the  Evaluator  Course,  to 
prepare  them  to  evaluate  the  performance  of  unit  instructors 
operating  as  aircrews  during  instructor  level  crew  coordination 
training.  Focusing  on  only  the  evaluation  aspect  of  the  course 
design  was  marginally  effective.  Although  the  DES  SIPs  were 
capable  of  evaluating  crew  coordination  skills,  their  lack  of 
exposure  to  the  complete  training  syllabus  threatened  to 
undermine  the  credibility  of  the  crew  coordination  program. 


Four  instructor  pilots  (IPs)  and  four  unit  trainers  (UTs) 
received  training  to  present  the  Aircrew  Coordination  Student 
Course  and  to  conduct  the  pre-  and  post-training  evaluations  and 
training  missions. 

Fifteen  battle  rostered  aircrei'/s  consisting  of  a  pilot  and  a 
copilot  gunner  participated  in  the  training.  Seven  of  the  15 
crews  included  a  unit  IP  or  UT  crew  coordination  instructor. 
Additionally,  unit  observation  and  utility  helicopter 
crewmembers,  both  rated  and  nonrated,  attended  the  classroom 
instruction.  Three  observation  helicopter  instructors  and  four 
observation  helicopter  aircrews  participated  in  testing  crew 
coordination  evaluation  procedures  in  the  actual  aircraft 
(Zeller,  1993) . 
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Materials 


Both  the  training  and  evaluation  components  of  the  Aircrew 
Coordination  Exportable  Training  Package,  December  1992  were 
implemented  as  published. 


The  training  package  is  designed  for  three  audiences: 
trainers,  instructors,  and  aircrews.  USAAVNC  certified  crew 
coordination  trainers  teach  unit  instructors,  who  then  train  unit 
aircrews,  both  rated  and  nonrat ed  crewmembers.  Training  for  the 
unit  instructors  consisted  of  26  hours  of  classroom  instruction, 
four  5-hour  missions  in  the  AH-64  combat  mission  simulator  (CMS), 
and  five  hours  of  practice  evaluating  crew  performance. 

Classroom  instruction  for  unit  instructors  covered  methods  of 
instruction,  the  Aircrew  Coordination  Student  Course,  and 
scenario  development  and  evaluation  procedures.  Training  for 
unit  aircrews  consisted  of  18  classroom  hours  and  four  5-hour 
tactical  missions  in  the  CMS.  Simulator  missions  included 
premission  planning  and  rehearsal,  simulator  flight,  crew  level 
after-action  review,  and  an  instructor  debriefing. 

The  AH-64  CMS  is  a  fixed-base  simulation  system  designed  for 
training  in  the  use  of  AH-64  Apache  helicopters.  The  CMS 
simulates  the  Apache  helicopter  and  its  related  systems  to  the 
same  level  of  performance  as  found  in  the  operational  systems. 

The  CMS  consists  of  two  separate  compartments  for  the  pilot  and 
copilot  gunner,  each  having  a  six-degree-of-freedom  hydraulic 
motion  system.  Each  compartment  includes  a  crewmember  station, 
pilot  or  copilot  gunner,  in  the  forward  portion  and 
instructor/operator  and  observer  stations  in  the  rear  portion. 
Each  compartment  is  equipped  with  a  visual  system  that  simulates 
natural  helicopter  environment  surroundings.  The  CMS  provides 
normal  and  emergency  procedural  mission  training  and  weapons 
delivery.  Additional  capabilities  include  navigation,  instrument 
flight  operation,  day,  dusk,  and  night  visual  flight  operations, 
ordnance  delivery  and  aircraft  survivability  systems  of  the 
attack  helicopter.  In  addition.  The  CMS  can  be  used  to  simulate 
tactical  threat  systems. 

Each  simulator  mission  was  recorded  using  four  video  cameras 
multiplexed  onto  one  video  picture.  All  intercom  and  radio 
communications  and  aural  warnings  were  recorded  onto  the 
videotape.  One  camera  was  placed  in  each  simulator  compartment 
and  aimed  to  provide  a  high-over-the-shoulder  view  of  each 
crewmember.  Two  cameras  were  placed  in  the  project  control  room 
and  aimed  at  monitors  to  capture  each  crewmember's  visual  field 
of  view  and  symbology  overlay. 

Evaluation  procedures  are  an  integral  part  of  the  training 
package  for  unit  instructors.  Measures  similar  to  those 
previously  used  for  the  utility  helicopter  validation  testbed 
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were  developed  to  test  data  collection  techniques  using  the  AH-64 
CMS.  The  primary  question  asked  through  the  use  of  the 
evaluation  measures  was  whether  the  aircrews  showed  improvement 
between  the  pre-  and  post-training  evaluations.  The  measures  and 
the  results  of  their  use  in  the  attack  helicopter  training  are 
described  in  this  report.  Following  is  a  list  of  measures  used 
in  the  training.  Measures  that  are  part  of  the  training  package 
are  italicized. 

1.  Attitude.  The  "Army  Aviation  Crewmember 
Questionnaire" . 

2.  Behavior.  Basic  Qualities  associated  with  crew 
coordination  captured  through  the  use  of  the  Aircrew  Coordination 
Training  Grade  Slip. 

3.  Behavior  and  Performance.  A  grade  slip  based  on  AH-64 
ATM  tasks  revised  to  incorporate  aircrew  coordination 
considerations . 

4.  Crew  Mission  Performance.  Measures  of  crew  performance 
made  during  tactical  scenario  execution. 

5.  Participant  Exit  Interviews.  A  form  for  debriefing 
participants  at  the  conclusion  of  the  training. 

Four  attack  helicopter  tactical  scenarios  were  developed  for 
the  crew-level  evaluations  to  assess  changes  in  crew  mission 
performance  and  training  missions  to  apply  classroom  instruction. 
The  baseline  evaluation  was  conducted  prior  to  the  crew 
coordination  training.  The  second  evaluation  was  administered 
after  the  training.  Crew  exposure  to  the  evaluation  scenarios 
was  counterbalanced,  that  is,  crews  given  evaluation  scenario  one 
for  the  pretraining  mission  were  given  scenario  two  for  the  post¬ 
training  mission  and  vice  versa.  The  objectives  and  tasks 
incorporated  into  the  two  evaluation  scenarios  were  made  to 
present  two  equally  difficult  missions  to  the  aircrews.  Two 
training  scenarios  followed  the  classroom  instruction  and 
preceded  the  post-training  evaluation.  Figure  1  shows  how 
training  scenarios  one  and  two  provided  step  increases  in 
difficulty  leading  to  the  post-training  evaluation  mission.  This 
progression  in  mission  difficulty  supported  the  Army's  "crawl- 
walk-run"  training  philosophy. 
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Difficulty 


Pretraining  ■* - ►  Post-training 

Evaluation  Counterbalance  Evaluation 


Training 

Two 


Training 

One 


Sequence 


Figure  1.  Evaluation  and  training  scenarios 
Procedures 


The  training  schedule  is  presented  in  Table  1.  The  first 
step  in  the  training  was  for  project  staff  to  instruct  DES  SIPs 
to  assist  in  evaluating  the  unit  instructor  evaluation  and 
training  missions.  Project  staff  and  DES  then  instructed  and 
evaluated  the  participating  IPs  and  UTs. 

After  receiving  instruction,  the  IPs  rated  the  15  aircrews 
during  a  full  (premission,  flight,  post-mission)  simulator 
mission.  This  was  the  pretraining  evaluation  designed  as  the 
baseline  against  which  performance  improvements  would  be 

measured.  The  15  attack  helicopter  crews  were  divided  into  two 
sections  for  the  classroom  instruction.  Utility  helicopter  crews 
and  battalion  staff  members  were  added  to  one  of  the  academic 
sections.  A  third  academic  section  was  formed  for  observation 
helicopter  aircrews.  Instructor  teams  consisting  of  unit  IPs  and 
UTs,  were  formed  to  team-teach  the  classroom  instruction.  The 
instructor  teams  also  instructed  the  aircrews  during  the  two 
training  missions.  Subsequent  to  the  training,  the  post-training 
evaluation  was  given  to  each  of  the  15  attack  helicopter 
aircrews . 

When  the  evaluation  missions  were  completed,  training 
participants  were  asked  to  participate  in  an  exit  interview  to 
critique  the  training. 
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Table  1 


Training  Schedule 


Activity  Date 

Evaluator  Training  12  -  14  May  $3 

•  DES  SIPs 

Instructor  Training  1-11  Jun  93 

•  IPs  and  UTs 


Pretraining  Evaluation  Mission 

•  AH-64  crews 

Aircrew  Training^ 

•  AH-64  crews 

•  OH-58  crews 

•  UH-60  crews 

Post-training  Evaluation  Mission 

•  AH-64  crews 

Participant  Exit  Interviews^ 

^  Includes  battalion  staff  personnel 


14  -  16  Jun  93 

18  -  30  Jun  93 


1-7  Jul  93 

8-9  Jul  93 


Results  of  the  Training 

This  section  summarizes  the  results  from  the  pre-  and  post¬ 
training  evaluations  in  terras  of  the  measures  described  above. 

Attitudes 


The  Army  Aviation  Crewmember  Questionnaire  includes  46 
statements  for  which  aviators  are  asked  to  rate  the  extent  of 
their  agreement  or  disagreement  on  a  seven-point  scale.  Also 
included  in  the  questionnaire  is  a  section  used  to  collect 
background  data  from  the  respondents.  The  current  version  of  the 
Army  Aviation  Crewmember  Questionnaire  is  presented  in  Appendix 
A. 


The  Army  Aviation  Crewmember  Questionnaire  is  used  to  assess 
three  primary  attitudes  associated  with  crew  coordination. 

1.  Communication  and  Coordination.  An  orientation  toward 
interpersonal  awareness,  communication,  and  crew  coordination. 

2.  Shared  Leadership.  An  attitude  toward  the 
appropriateness  of  sharing  responsibility  for  leadership. 
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3.  Recognition  of  Stressor  Effects.  An  attitude  accepting 
that  human  performance  is  affected  by  external  events  and 
allowance  must  be  made  for  changed  performance. 

The  questionnaire  was  administered  twice:  once  at  the  end 
of  the  pretraining  evaluation  mission  and  a  second  time  at  the 
end  of  the  post-training  evaluation  mission.  Scale  scores  were 
computed  for  the  3  attitude  areas  and  a  total  score  was  computed 
for  all  46  items.  The  twenty-three  negatively  worded  items  were 
recoded  so  that  a  7  represented  the  most  desirable  answer,  that 
is,  1=7,  2=6,  3=5,  5=3,  6=2,  and  7=1.  Average  item  scores  for 
each  scale  were  computed  for  the  pre-  and  post-training  missions. 
Table  2  shows  the  means  of  the  pre-  and  post-training  results  and 
whether  the  difference  between  the  two  is  statistically 
significant . 

Table  2 

Comparison  of  Mean  Item  Scores  for  Pre-  and  Post-training 
Administrations  of  the  Army  Aviation  Crewmember  Questionnaire 

IP/UT's  and  Aviators  {n=30) 


Attitude  scale 

Pre¬ 

training 

Post¬ 

training 

Communication  and 
Coordination 

5.86 

5.81 

Shared  Leadership 

5.55 

5.45 

Recognition  of  Stressor 
Effects 

4.66 

4.67 

Whole  Questionnaire 

5.53 

5.47 

Crewmembers  participating  in  the  training  generally  had  a 
favorable  attitude  toward  crew  coordination  in  both  the  pre-  and 
post-training  evaluations.  There  were  no  statistically 
significant  changes  between  the  pre-  and  post-training  results. 

Behaviors 


Crew  behaviors  are  the  principal  means  to  teach  and  evaluate 
crew  coordination  in  the  Aircrew  Coordination  Exportable  Training 
Package.  Based  on  previous  research,  the  USAAVNC  defined  crew 
coordination  behaviors  as  13  Basic  Qualities.  Each  Basic  Quality 
was  designed  to  be  rated  by  an  IP-evaluator  on  a  seven-point 
scale.  The  seven-point  scale  was  anchored  at  the  1,  4,  and  7 
levels  with  specific  behavioral  descriptions  of  performance  at 
those  levels.  IP-evaluators  were  instructed  to  interpolate 


7 


ratings  of  2,  3,  5,  and  6  from  the  descriptions  given  at  the  1, 
4,  and  7  levels  as  being  somewhat  better  or  worse  than  the 
anchored  description.  The  numbers  associated  with  the  Basic 
Quality  ratings  were  1  =  Very  Poor,  2  =  Poor,  3  =  Marginal,  4  = 
Acceptable,  5  =  Good,  6  =  Very  Good,  and  7  =  Superior.  IP- 
evaluators  provided  the  Basic  Quality  ratings  on  a  grade  slip 
designed  for  the  training  package.  The  Basic  Quality 
definitions,  anchor  descriptions,  and  rating  guidelines  are 
provided  in  Appendix  B. 


Table  3  shows  a  comparison  of  Basic  Quality  item  means 
between  pre-  and  post-training  evaluations  for  the  eight  crews 
without  an  IP  or  UT  crewmember.  There  was  improvement  in  every 
Basic  Quality.  Despite  the  small  sample  size,  a  statistically 
significant  improvement  was  reached  on  6  of  the  13  Basic 
Qualities.  On  average,  crews  moved  from  a  rating  of  "marginal" 
to  "acceptable"  within  the  timeframe  of  the  training. 
Understanding  that  crew  coordination  requires  practice,  the  goal 
of  the  initial  training  was  to  achieve  a  rating  of  "acceptable." 
Continuation  training  is  required  to  move  beyond  the  "acceptable" 
level  of  performance. 


Behavior  and  Performance 


ATM  tasks  contained  in  the  ATM  for  the  AH-64,  TC  1-214 
(Department  of  the  Army,  1992,  May)  were  used  as  a  measure  of 
behavior  and  performance  during  the  training.  All  ATM  tasks 
include  both  a  crew  coordination  and  a  technical  flight  skill 
component.  IP-evaluators  rated  crews  on  various  ATM  tasks  but 
were  required  to  rate  crews  on  25  selected  tasks  for  each 
evaluation  mission.  The  25  tasks  were  selected  for  the  following 
reasons : 

1.  They  provided  a  common  means  for  comparing  crew  pre- 
and  post-training  evaluation  missions. 

2.  They  are  crew  coordination  intensive. 

3.  The  25  tasks  were  represented  in  the  evaluation 
scenarios . 

Two  forms  were  used  to  record  ATM  task  and  Basic  Quality 
performance:  (a)  DA  Form  7121-R,  Mar  92,  entitled  Battle- 

Rostered  Crew  Evaluation/Training  Grade  Slip  and  (b)  the  Aircrew 
Coordination  Training  Grade  Slip  based  on  DA  Form  5882-R,  Mar  92, 
entitled  Maneuver/Procedure  Grade  Slip  for  AH-64  Aviators.  For 
purposes  of  this  report,  these  two  forms  are  collectively 
referred  to  as  the  grade  slip.  Blank  grade  slips  are  provided  in 
Appendix  C. 
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Table  3 


Basic  Quality  Comparisons  between  Pre-  and  Post-training 
Evaluation  Missions 


Crews  {n=8) 


Basic  Quality 

Pre¬ 

training 

Post¬ 

train 

ing 

1 

Establish  and  maintain  flight  team 
leadership  and  crew  climate 

4.50 

4.88 

2 

Premission  planning  and  rehearsal 

3.88 

4.25 

3 

Selection  of  appropriate  decision 
making  techniques 

3.50 

3.63 

4 

/ 

prioritize  actions  and  distribute 
workload 

3.38 

★  * 

4.13 

/ 

5/ 

Management  of  unexpected  events 

3.13 

-k 

4.13 

/6 

/ 

/ 

r\ 

Statements  and  directives  clear, 
timely,  relevant,  complete,  and 
verified 

3.25 

k  k 

4.88 

Maintenance  of  mission  situational 
awareness 

3.13 

3.88 

8 

Decisions  and  actions  communicated  and 
acknowledged 

3.63 

4.25 

9 

Supporting  information  and  actions 
sought  from  crew 

3.63 

k 

4.38 

10 

Crewmember  actions  mutually  cross 
monitored 

2.88_ 

kk 

4.00 

11 

Supporting  information  and  actions 
offered  by  crew 

3.50 

k 

4.38 

12 

Advocacy  and  assertion  practiced 

3.38 

4.13 

13 

Crew-level  after-action  reviews 
accomplished 

4.38 

4.63 

All  13  Basic  Quality  Ratings 

3.54 

kk 

4.27 

*p  i  .05  **p  5  .01^ 

^For  this  table  and  subsequent  tables,  significant  change 
(*)  indicates  a  95%  probability  that  the  results  from  the  sample 
are  what  would  occur  if  the  entire  population  had  completed  the 
training.  Highly  significant  change  (**)  equates  to  a  99%  prob¬ 
ability  that  the  sample  results  reflect  the  entire  population. 
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IP-evaluators  graded  ATM  task  performance  on  a  four-point 
scale:  S+,  S,  S-,  and  U.  When  a  crew  received  a  grade  of  S-  or 

U  due  to  crew  coordination,  IP-evaluators  noted  two  Basic 
Qualities  contributing  to  the  task  grade.  Basic  Quality  ratings 
are  recorded  on  page  2  of  the  Aircrew  Coordination  Training  Grade 
Slip.  IP-evaluators  provided  written  comments  and  an  overall 
grade  for  the  flight  on  the  Battle-Rostered  Crew 
Evaluation/Training  Grade  Slip.  The  ATM  task  grades  were 
translated  for  computer  analysis  so  that  S+  =  3,  S  =  2,  S-  =  1, 
and  U  =  0 . 

Table  4  shows  a  comparison  between  each  of  the  25  ATM  task 
grades  and  the  overall  grade  for  the  flight  on  the  pre-  and  post¬ 
training  evaluation  missions.  The  table  also  shows  a  comparison 
between  the  average  task  grade  for  all  25  ATM  tasks.  It  is 
noteworthy  that  performance  for  24  of  the  25  ATM  tasks  improved. 
Performance  for  one  ATM  task  remained  the  same.  Although  a 
sample  this  small  is  unlikely  to  render  statistically  significant 
results,  nine  of  the  comparisons  represent  significant 
improvement  from  pre-  to  post-training  scores. 

The  average  score  for  the  25  ATM  tasks  improved 
significantly.  Before  the  training,  the  crews  were  able  to 
obtain  an  S/S-  level  of  performance.  After  the  training,  crews 
attained  an  S  level  of  performance.  As  a  group,  task-level 
performance  improved  by  one-half  a  grade  subsequent  to  the 
training.  The  overall  grade  for  the  flight,  a  grade  composed  of 
technical  flying  skill,  crew  coordination,  and  mission 
effectiveness,  also  improved  by  one-half  a  grade:  on  the 
pretraining  evaluation,  the  average  grade  was  slightly  above  S-, 
whereas  after  the  training,  the  overall  grade  was  nearly  S. 
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Table  4 


Aviators  (n=8)  GROUPED  T-Tests 

ATM  Task  Grade^  Comparisons  between  Pre-  and  Post-training 
Evaluations 


AH- 64  ATM  Task 

Valid 

n 

Pre- 

training 

Valid 

n 

Post¬ 

training 

1000 

Crew  Mission  Briefing 

8 

1.75 

★ 

8 

2.25 

1004 

DA  Form  5701-R  (PC) 

8 

1.88 

8 

2.13 

1007 

Engine-start,  run-up,  hover,  and 
before  take-off  checks 

8 

1.75 

7 

2.00 

1016 

Hover  Power  Check 

8 

1.00 

8 

2.00 

1023 

Fuel  Management  Procedures 

8 

1.13 

* 

7 

2.14 

1026 

Doppler  Navigation 

8 

1.75 

7 

2.14 

91033 

Terrain  Flight  Mission  Planning 

7 

1.86 

8 

2.13 

1034 

Terrain  Flight  Takeoff 

8 

1.25 

8 

2.13 

1035 

Terrain  Flight 

8 

1.25 

★ 

8 

1.88 

1038 

Terrain  Flight  Approach 

7 

1.29 

■k 

7 

2.14 

1064 

Terrain  Flight  Navigation 

8 

1.75 

8 

2.00 

1068 

Emergency  Procedures 

8 

1.50 

■k 

8 

2.25 

1076 

Radio  Navigation 

3 

1.33 

5 

1.80 

1081 

Nonprecision  Approach 

3 

0.33 

k 

5 

1.40 

1083 

IMC  Procedures /VHIRP 

3 

1.33 

5 

1.60 

1090 

Masking  &  Unmasking 

5 

1.40 

8 

1.63 

1095 

A/C  Survivability  Equipment 

8 

1.63 

8 

2.13 

1119 

Firing  Position  Ops 

8 

1.75 

8 

1.75 

1140 

Engage  Tgt  w/Hellfire 

8 

1.38 

8 

1.63 

1141 

Engage  Tgt  w/Arcs 

1 

1.00 

— 

3 

2.00 

1142 

Engage  Tgt  w/AWS 

1 

0.00 

— 

2 

2.00 

2008 

Evasive  Maneuvers 

8 

1.25 

7 

1.86 

2043 

FARP  Procedures 

7 

1.86 

6 

2.00 

2050 

Select  Appropriate  Weapon  System 

8 

1.75 

8 

1.88 

2052 

ID  Targets  w/TADS 

8 

1.88 

8 

2.13 

Average  Score  for  the  25  Tasks 

8 

1.50 

** 

8 

1.97 

Overall  Grade  for  Flight 

8 

1.25 

8 

1.75 

*p  5  ,05  ^  .01  —  could  not  be  perform  test:no  variance  or  too  few  cases 


^  (U=0,  s-=l,  S=2,  S+=3) 
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When  an  ATM  task  was  graded  S-  or  U  and  the  grade  involved 
crew  coordination,  IP-evaluators  noted  two  Basic  Qualities 
contributing  to  the  grade.  Table  5  shows  which  ATM  tasks  were 
problems  and  which  Basic  Qualities  contributed  to  less  than 
satisfactory  task  performance.  Looking  down  the  columns  in  Table 
5,  there  were  improvements  in  the  following  Basic  Qualities: 

BQ3  Application  of  appropriate  decision  making  techniques 
BQ4  Prioritize  actions  and  distribute  workload 
BQ5  Management  of  unexpected  events 

BQ6  Statement  and  directives  clear,  timely,  relevant,  complete, 
and  verified 

BQ7  Maintenance  of  mission  situational  awareness 

BQ8  Decisions  and  actions  communicated  and  acknowledged 

BQ9  Supporting  information  and  actions  sought  from  crew 

BQIO  Crewmember  actions  mutually  cross-monitored 

BQll  Supporting  information  and  actions  offered  by  crew 

BQ12  Advocacy  and  assertion  practiced 

The  following  Basic  Qualities  continued  to  contribute  to 
less  than  satisfactory  task  performance  or  improved  only 
slightly.  These  Basic  Qualities  should  receive  increased 
emphasis  from  crew  coordination  trainers  and  instructors: 

BQ2  Premission  planning  and  rehearsal 

BQ3  Application  of  appropriate  decision  making  techniques 
BQ7  Maintenance  of  mission  situational  awareness 
BQIO  Crewmember  actions  mutually  cross-monitored 

Looking  across  the  rows  in  Table  5,  one  can  see  that  crews 
continue  to  struggle  with  certain  ATM  tasks  even  after  crew 
coordination  training.  Although  Table  4,  the  ATM  Task  Grades, 
also  shows  which  ATM  tasks  were  problematic.  Table  5  presents 
more  precise  information  because  it  is  linked  to  the  Basic 
Qualities.  The  following  is  a  list  of  the  ATM  tasks  that 
continue  to  cause  problems  after  the  training,  that  is,  those 
tasks  with  four  or  more  Basic  Quality  negative  notations: 

1081  Perform  nonprecision  approach 
1083  Perform  inadvertent  IMC  procedures/VHIRP 
1090  Perform  masking  and  unmasking 
1119  Perform  firing  position  operations 
1140  Engage  target  with  Hellfire 

Collectively,  these  five  tasks  accounted  for  47  (35%)  of  the 
negative  notations.  This  type  of  information  should  alert  the 
unit  to  the  ATM  tasks  that  need  to  be  emphasized  in  the 
commander's  crew  coordination  training  program. 
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Less  Than  Satisfactory  ATM  Task  Performance  and  Inhibiting  Basic  Qualities 
for  Pre-  and  Post-training  Evaluation*  (n=8) 


Totals  0|0  213  9|7  7|0  7|0  9|1  26|13  1|0  4|0  26|7  6|2  4|0  0|0  101|33 

♦Within  each  column,  numbers  to  the  left  are  pre-training  and  numbers  to  the  right  are  post-training. 

BQl=Crew  climate  BQ2-Plan  rehearse  BQ3=Decision  techniques  BQ4=Work  load  BQ5=Unexpected  events 

BQ6=Positive  comm  BQ7=Situation  awareness  BQ8=Comm/acknowledged  BQ9=Info  sought  BQ10=Cross-monitor 

BQll=Info  offered  BQ12=Advoc/assert  BQ13=After  Action  Reviews 


Crew  Mission  Performance 


Aircrews  were  given  attack  helicopter  tactical  missions  to 
perform  in  the  AH-64  combat  mission  simulator  (CMS) .  Data  was 
collected  from  the  two  evaluation  missions  (pre-  and  post¬ 
training)  to  determine  whether  mission  performance  was  enhanced 
as  a  result  of  the  training.  The  evaluation  scenarios  included  a 
mix  of  the  mission  segments  and  activities  shown  in  Figure  2. 


Mission  Segments 

Premission  pianning 
Forward  assembly  area  to 
rearm  and  refuel 

Rearm  and  refuel  to  holding  area 
Holding  area  to  battle  position 
Battle  position  to  battle  position 
Battle  position  to  rearm  and 
refuel/recovery  airfield 


Activities 


Navigation 

Threat  avoidance  and  evasion 
Minor/major  malfunction 
Occupy  battle  position 
Acquire,  identify,  and  engage  targets 
Unexpected  events 
Instrument  recovery 


Figure  2.  Evaluation  scenario  outlines 


Mission  performance  measures  are  not  required  in  the  Aircrew 
Coordination  Exportable  Training  Package.  The  purpose  of 
including  mission  performance  measures  in  the  attack  helicopter 
battalion  training  was  to  test  measures  and  data  collection 
procedures  for  possible  use  in  the  next  phase  of  the  project, 
that  is,  comparing  performance  of  battle  rostered  and  mixed 
aircrews.  Project  staff  measured  mission  performance  using  a 
variety  of  data  collection  techniques:  video  recordings,  live 
observation  data  log,  simulator  printouts,  and  evaluator 
worksheets.  Mission  performance  data  was  analyzed  in  the 
following  categories:  navigation,  weapons  employment,  threat, 
unexpected  events,  and  instrument  approach. 

Mission  Performance  -  Navigation.  This  performance  measure 
evaluates  the  crew's  ability  to  remain  within  altitude  and  course 
limitations  as  well  as  avoid  inadvertent  obstacle  strikes  and 
collisions  with  the  ground. 
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Overall,  performance  improved  between  the  pre-  and  post¬ 
training  evaluation  missions.  There  was  a  60%  reduction  in 
altitude  deviations  (8  to  3) ,  a  50%  reduction  in  course 
deviations  (4  to  2) ,  and  a  40%  reduction  in  obstacle/ground 
strikes  (10  to  6) .  Three  of  the  8  crews  evaluated  had  no 
altitude  deviations  while  5  crews  had  from  1  to  5  deviations  per 
mission.  Similarly,  3  crews  had  no  course  deviations  whereas  the 
other  5  crews  had  1  deviation.  One  crew  completed  both 
evaluation  missions  without  either  altitude  or  course  errors. 

Mission  Performance  -  Weapons  Employment.  This  performance 
measure  evaluates  the  crew's  ability  to  properly  identify, 
acquire,  and  accurately  engage  targets. 

The  8  crews  engaged  27%  more  targets  (32  versus  44),  shot 
23%  more  missiles  (41  versus  53) ,  and  scored  29%  more  hits  and 
kills  (29  versus  41)  with  the  same  number  of  misses  (12)  on  the 
post-training  mission.  Most  crews  engaged  targets  only  with 
missiles.  Four  crews  fired  rockets  and  one  crew  fired  the  gun 
during  the  16  missions.  Only  one  crew  engaged  targets  with  all 
three  weapon  systems.  No  rocket  engagements  hit  or  killed  a 
target.  The  crew  that  engaged  targets  with  the  gun  expended  100 
rounds  to  realize  4  hits  and  1  kill. 

Mission  Performance  -  Threat.  This  performance  measure 
involves  the  crew's  ability  to  avoid  and  successfully  evade 
threat  radar-controlled  weapon  systems.  It  also  describes 
detrimental  outcomes  resulting  from  threat  detection;  that  is, 

^he  crew  becomes  misoriented,  the  aircraft  is  hit,  or  the 
aircraft  crashes.  Threat  systems  used  for  both  evaluation 
scenarios  included:  ZSU-23-4,  SA6,  SA8,  and  Straight  Flush 
radars  as  well  as  T80  and  T72  tanks. 

All  8  crews  were  detected  by  threat  systems  during  both  the 
pre-  and  post-training  evaluation  missions.  The  average  number 
of  warnings  for  both  missions  was  6  with  a  range  of  1  to  20 
warnings  per  mission.  The  average  nxmber  of  warnings  decreased 
from  9  to  4  between  the  pre-  and  post-training  evaluations .  More 
than  57%  of  the  crews  had  fewer  threat  encounters  on  their  post¬ 
training  evaluation  mission.  There  were  9  detrimental  outcomes 
with  over  half  of  them  occurring  during  the  pretraining  mission. 

There  were  technical  difficulties  in  collecting  threat 
mission  performance  data.  To  support  this  measure  in  the  CMS,  a 
capability  designed  to  record  when  the  aircraft  has  line  of  sight 
with  a  target  was  used.  When  the  aircraft  was  within  range  of 
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the  threat  system’s  radar,  the  same  line  of  sight  criteria  was 
used  to  record  threat  detection.  However,  the  CMS  has  no 
capability  to  determine  or  record  the  threat  radar  operating 
mode;  that  is,  search,  track,  or  engage.  Also,  the  number  of 
threat  radar  warnings  and  the  audio  alert  from  the  aircraft  radar 
warning  receiver  confound  recording  the  length  of  each  warning. 
Additional  research  and  development  is  needed  to  improve  the 
precision  of  measuring  threat  related  crew  mission  performance  in 
the  CMS. 

Mission  Performance  -  Unexpected  Events.  This  performance 
measure  evaluates  the  crew's  ability  to  work  in  concert  while 
coping  with  emergencies,  malfunctions,  and  inadvertent  entry  into 
instrument  meteorological  conditions  (IMC) .  This  measure 
occurred  only  in  one  of  the  evaluation  scenarios. 

Crews  experienced  two  emergencies  in  evaluation  scenario 
number  one.  During  the  pre-training  evaluation,  60%  of  the  crews 
(3  of  5)  managed  the  emergencies  satisfactorily  while  100%  of  the 
crews  (3  of  3)  satisfactorily  performed  the  emergency  procedures 
in  the  post-training  evaluation.  The  two  unsatisfactory  crews 
during  the  pretraining  evaluation  experienced  a  hard  landing  and 
crash  respectively. 

During  evaluation  scenario  niomber  two,  crews  experienced  a 
500  pound  loss  of  fuel  after  departing  the  first  battle  position 
enroute  to  the  next  battle  position.  Three  crews  experienced 
this  malfunction  during  their  pretraining  evaluation.  Five  crews 
experienced  the  malfunction  during  the  post-training  evaluation. 
Data  was  not  recorded  for  one  of  the  post-training  evaluation 
crews.  Only  42%  of  the  crews  (3  of  7)  detected  the  fuel  leak;  1 
during  the  pretraining  evaluation  and  2  during  the  post-training 
evaluation . 

Also  in  evaluation  scenario  number  two,  crews  experienced 
deteriorating  weather  conditions  on  the  flight  back  to  the 
forward  arming  and  refueling  point  which  resulted  in  a  loss  of 
visual  reference  while  terrain  flying.  As  with  the  fuel  leak, 
three  crews  experienced  inadvertent  entry  into  instrument 
meteorological  conditions  (IMC)  during  their  pretraining 
evaluation.  Five  crews  experienced  IMC  during  their  post¬ 
training  evaluation.  Only  one  crew's  performance  of  recovery 
procedures  was  less  than  acceptable  resulting  in  a  crash  during  a 
pretraining  evaluation.  There  were  no  crews  who  performed  poorly 
during  the  post-training  evaluation. 
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Mission  Performance  -  Instrument  Approach.  This 
performance  measure  evaluated  the  crew's  ability  to  plan  and 
execute  a  non-precision  instrument  recovery  procedure  after 
inadvertently  entering  IMC. 

Coupled  with  the  IMC  unexpected  event,  this  measure  occurred 
only  during  evaluation  scenario  number  two.  Three  dimensions  of 
this  measure  were  observed  for  each  crew:  planning  the  approach, 
timing  the  inbound  leg  of  the  approach,  and  the  outcome  of  the 
approach.  Pre-  and  post-training  evaluation  results  are  combined 
because  of  imprecise  data  collection  procedures  and  less  than 
satisfactory  crew  performance  of  the  ATM  task  overall.  Less  than 
50%  of  the  crews  (3  of  7)  recorded  during  pre-  and  post-training 
evaluations  adequately  planned  the  approach.  Only  1  crew  of  the 
6  crews  recorded  during  pre-  and  post-training  evaluations 
properly  timed  the  approach.  Two  of  the  three  pretraining 
evaluations  resulted  in  unsuccessful  approaches,  that  is,  missed 
approach  and  airfield  not  in  sight. 

Participant  Exit  Interviews 

Instructors  were  interviewed  after  the  last  day  of  the 
training.  Prior  to  the  exit  interview,  instructors  and 
crewmembers  were  given  the  interview  questions  to  make  notes  on 
the  items  they  wanted  to  discuss.  Instructors  and  many  of  the 
crewmembers  wrote  answers  to  the  interview  questions  and  gave 
them  to  the  project  staff.  Instructors  were  interviewed  as  a 
group  during  a  two-hour  session.  The  project  staff  recorded  the 
group's  responses  and  discussions.  After  all  the  interview  data 
were  collected,  that  is,  participant  written  responses  and 
project  staff  notes,  the  responses  were  entered  into  computer 
files,  edited  for  readability,  and  compiled  to  eliminate 
duplicate  responses.  The  results  of  the  analysis  are  presented 
in  Appendix  D.  Following  is  a  summary  of  the  suggested 
improvements  from  the  instructor  exit  interviews: 

1 .  Instructors  should  receive  the  Student  Course  block  of 
instruction  with  their  battle-rostered  crewmember.  Participating 
as  a  crewmember  after  receiving  the  Instructor  Course  tends  to 
degrade  the  battle-rostered  crewmember's  training. 

2.  The  methods  of  instruction  block  should  be  the  last 
subject  in  the  Instructor  Course.  Emphasis  should  be  on 
instructional  techniques  and  main  points  of  instruction  for  each 
syllabus  hour. 
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3.  Instructors  should  be  required  to  "teach  back"  the 
material  to  the  trainers  as  a  check  on  the  new  instructor's 
teaching  ability. 

4.  When  the  course  is  exported,  USAAVNC  should  consider 
monitoring  the  new  instructor's  first  training  class  for  content 
accuracy  and  presentation  technique. 

5.  Video  tapes  of  the  flight  are  essential  for  effective 
training  and  evaluation. 

Conclusions  and  Recommendations 


The  crew  coordination  training  of  AH-64  attack  helicopter 
battalion  crews  at  Fort  Rucker,  AL  was  very  effective.  The  data 
collected  show  that  the  crew  coordination  training  and  evaluation 
system  positively  affects  behavior,  enhances  mission  performance, 
and  increases  the  margin  of  safety.  The  positive  results 
reported  in  this  paper  are  comparable  to  the  results  achieved  by 
UH-60  crews  in  the  1992  Crew  Coordination  Exportable  Training 
Package  validation  testbed.  For  example,  the  ATM  task  average 
grade  for  both  types  of  crews  improved  one-half  grade  between 
pre-  and  post-training  evaluations.  Basic  Qualities  7, 
Maintenance  of  Mission  Situational  Awareness,  and  10,  Crewmember 
Actions  Mutually  Cross-monitored,  made  the  largest  contribution 
to  decreasing  less-than-satisfactory  ATM  task  performance. 

Crew  performance  improvements  in  the  navigation  measure, 
especially  obstacle  strikes  and  ground  collisions,  and  the 
weapons  employment  measure  confirmed  the  effectiveness  of  both 
the  training  itself  and  the  trial  measures  and  data  collection 
procedures.  All  five  mission  performance  measures  provided 
meaningful  data.  As  described  in  the  report,  there  were  a  number 
weaknesses  in  the  trial  measures  that  require  improvement  prior 
to  the  data  collection  phase  of  the  interaction  of  battle¬ 
rostering  and  crew  coordination  research  project.  Also,  there 
were  suggested  improvements  to  the  course  of  instruction  that 
warrant  consideration.  Following  is  list  of  recommendations  for 
both  the  course  of  instruction  and  evaluation  measures  and  data 
collection  techniques. 

1.  Avoid  attempts  to  abbreviate  the  Aircrew  Coordination 
Exportable  Training  Package  to  meet  specific  project  or  target 
audience  needs.  For  example,  the  abbreviated  Evaluator's  Course 
to  train  a  group  of  IP-evaluators  marginally  prepared 
participants  to  evaluate  crew  performance  while  failing  to 
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provide  sufficient  understanding  of  the  fundamentals  of  crew 
coordination.  Efforts  such  as  this  could  potentially  jeopardize 
the  entire  crew  coordination  program,  because  in  this  situation 
the  recipients  were  a  highly  influential  group  of  DES  SIP 
aviators . 

2.  Develop  options  for  implementing  the  Aircrew 
Coordination  Exportable  Training  Package  that  allow  trainers  and 
instructors  to  change  the  sequence  and  format  of  selected  blocks 
of  instruction.  For  example,  develop  materials  to  support 
teaching  the  methods  of  instruction  block  last,  include 
participant  teach-back  of  materials,  exchange  an  evaluation 
mission  of  an  operational  crew  for  reviewing  video  segments  as  a 
means  to  practice  evaluation  techniques,  and  include  an 
administrative  support  plan  checklist  for  new  instructors  to 
coordinate  required  resources  to  teach  other  instructors  and/or 
unit  aircrews. 

3.  Discontinue  use  of  the  Army  Aviation  Crewmember 
Questionnaire  as  a  research  tool.  Little  useful  information  will 
be  gained  through  its  use. 

4.  Refine  mission  performance  measure  descriptions  and 
criteria  to  exploit  CMS  capabilities.  For  example,  develop 
efficiency  ratios  for  missile  engagements. 

5.  Research  CMS  capabilities  to  provide  more  precise  threat 
measurement  data.  For  example,  identify  methods  to  collect  type 
of  threat  warning  and  duration  measures. 

6.  Analyze  data  collection  sources  and  workload 
requirements  to  prepare  a  data  collection  plan  that  provides  a 
cross-check  among  sources  without  burdening  any  one  collection 
source  or  requiring  extensive  review  of  mission  video  tapes.  For 
example,  determine  specific  collection  tasks  for  IP-evaluator 
using  evaluator  worksheets,  CMS  console  operator  using 
preprogrammed  report  formats,  and  live  observation  data 
collectors  using  data  logger  software. 

7 .  Schedule  and  control  scenarios  and  crews  to  provide 
valid  comparisons  of  crew  performance  across  scenarios  and 
conditions.  For  example,  ensure  that  crew  exposure  to  scenarios 
with  malfunctions  and  IMC  can  be  compared  within  groups  without 
having  to  repeat  every  condition  in  each  scenario. 
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Appendix  A 


Rev.  6 


Army  Aviation  Crewmember  Questionnaire 
Instructions 


The  US  Army  Aviation  Center  (USAAVNC)  and  the  US  Army  Research  Institute  (ARI)  are 
researching  the  area  of  crew  coordination  in  Army  Aviation.  The  goal  of  this  research  is  to 
improve  performance  and  increase  the  margin  of  safety  on  an  Army-wide  basis.  Previous  research 
in  the  area  of  crew  coordination  has  contributed  to  substantial  gains  in  both  performance  and  safety. 

Because  Army  Aviation  is  unique,  much  of  the  information  discovered  by  other  services  and  the 
commercial  world  is  not  directly  applicable  to  the  Army  Aviation  environment.  Consequently,  the 
USAAVNC-ARI  research  program  is  designed  to  meet  the  specific  needs  of  Army  aviation.  As  a 
result  of  this  approach,  the  following  actions  are  now  ongoing  or  planned:  Mission  simulations  are 
being  developed  to  stress  aircrew-type  tasks,  enhanced  aircrew  coordination  training  is  being 
deployed,  the  US  Army  Safety  Center  is  incoiporating  crew  factors  into  the  accident  investigation 
process,  new  Aircrew  Training  Manuals  have  been  written,  the  annual  proficiency  and  readiness 
test  program  is  being  revised,  and  revisions  to  readiness  reporting  are  being  planned. 

This  Army  Aviation  Crewmember  Questionnaire  has  been  developed  as  part  of  the  USAAVNC-ARI 
research  program  to  obtain  your  opinion  about  crew  operations.  As  an  Army  aviator,  your 
participation  is  essential  to  the  program’s  success.  Your  opinions  are  important  and  will  be  used 
to  guide  the  next  phase  of  the  research  program. 

The  questionnaire  should  take  approximately  20  minutes  to  complete.  The  first  page  of  the 
questionnaire  asks  you  for  background  information  —  please  try  to  be  accurate.  The  next  three 
pages  contain  46  statements  for  which  there  are  no  "right"  or  "wrong"  answers.  We  are  simply 
asking  for  your  honest  opinion  to  each  statement.  Please  consider  each  statement  carefully. 

THANK  YOU  FOR  YOUR  PARTICIPATION. 


IMPORTANT 

The  information  you  provide  in  this  questionnaire  is 
confidential  and  will  be  used  for  research  purposes 
only.  Your  answers  will  neither  be  attributed  to  you 
personally  nor  become  a  part  of  any  personnel  or 
aviation  record  kept  on  you. 


Army  Aviation  Crevraiember  Questionnaire 

.  Background  Information 

(Please  complete  the  following  information  regarding  your  personal  experiences  and  current  status.) 

1 .  Aviation  Experience: 

(including  visual  flight  Lifetime  Flying  Experience  Experience  over  last  6  months 

simulator  hours) 

All  NV  Devices  All  NV  Devices 

Conditions  (e.g.,  NVG)  Conditions  (e.g.,  NVG) 

a.  Primary  acft  hrs.  _  _  _  _ 

b.  R/W  hrs.  _  _ _  _  _ 

c.  Fixed  Wing  hrs.  _  _  _  _ 

2.  Primary  Aircraft _ (Fill  in  aircraft  designation) 

3.  Current  Rank _ 

4.  Current  Unit  (Co/Bn/Rgt) _ 

5.  Time  in  Current  Unit  (months) _ 

6.  Current  Aviator  Readiness  Level  (RL)  1  2  3  FAC  (circle  one) 

7.  Current  Crew  Readiness  Level  (CRL)  1  2  (circle  one) 

8.  Current  primary  duty  assignment  in  unit  (circle  one): 

PC*  PI  CP  CPG  CE/FE  AO/AFSO/TO  OR* 

*^Note:  PC  includes  IP,  SP,  IE,  UT,  ME,  MP  duty  positions;  OR  includes  gunner  and  flight  medic. 

9.  Have  you  previously  had  Aircrew  Coordination  Training?  Yes  or  No  (If  yes,  answer  below.) 

Course  Title(s) _  Approximate  Date(s) _ 

10.  Are  you  .battle  rostered  with  another  crewmember?  Y  or  N  (circle  one). 

If  yes,  for  how  long?  _ months. 

Approximately  how  many  hours  have  you  flown  in  the  last; 


a. 

30  days 

battle  rostered  hours 

total  hours 

b. 

60  days 

battle  rostered  hours 

total  hours 

c. 

90  days 

battle  rostered  hours 

total  hours 

d. 

1 80  days 

battle  rostered  hours 

total  hours 

1 1 .  Cross-indexing  Code  (Note:  Because  the  results  of  this  questionnaire  will  be  correlated  with  other 
measures,  a  social  security  number  is  required.) 


Social  Security  §: 


Today’s  Date 


(day/mo/yr) 


Rev.  6 


II.  Opinion  Survey 

(Please  circle  the  number  on  the  agree-disagree  dimension  that  best  reflects  vour  personal  attitude. 
toward  each  statement.  There  are  no  "right "  or  "wrong "  answers.  We  are  simply  asking  for  your 
honest  opinions.) 


Slrtmgly  Disagree  Slightly  Neutral  Slightly  Agree  Strongly 
Disagree  Disagree  Agree  Agree 


1 .  Crewmembers  should  feel  obligated  to  mention  their  own  1 

psychological  stress  or  physical  problems  to  other  crewmembers 
before  or  during  a  mission. 

2.  Crewmembers  should  monitor  each  other  for  signs  of  stress  or  1 
fatigue  and  should  discuss  the  situation  with  the  affected 
crewmember(s). 

3.  Good  communication  and  crew  coordination  are  as  important  1 
as  technical  proficiency  for  the  safety  of  the  flight. 

4.  Crewmembers  should  be  aware  of  and  sensitive  to  the  personal  1 
problems  of  other  crewmembers. 

5.  The  pilot  flying  the  aircraft  should  verbalize  plans  for  1 

procedures  or  maneuvers  and  should  be  sure  that  the 
information  is  understood  and  acknowledged  by  affected 
crewmembers. 

6.  Even  when  fatigued,  I  perform  effectively  during  most  critical  1 
flight  maneuvers. 

7.  Pilots-in-command  should  encourage  pilots  and  crew  chiefs  to  1 
question  procedures  and  flight  profile  deviations  during  normal 
flight  operations  and  in  emergencies. 

8.  There  are  no  circumstances  where  the  pilot  should  take  the  1 

aircraft  controls  without  being  directed  to  do  so  by  the  pilot-in¬ 
command. 

9.  A  debriefing  and  after  action  review  of  procedures  and  1 

decisions  after  each  mission  are  important  for  developing  and 
maintaining  effective  crew  coordination. 


10.  Crew  coordination  is  more  important  under  high  stress  1 

conditions  than  It  is  under  low  stress  conditions. 

11.  Effective  crew  coordination  requires  crewmembers  to  take  into  1 
account  the  personalities  of  other  crewmembers. 

12.  The  pilot-in-command’s  responsibilities  include  coordinating  1 

inflight  crew  chief  activities. 

13.  Most  crewmembers  are  able  to  leave  personal  problems  behind  1 
when  flying  a  mission. 


2  3  4  5  6  7 


2  3  4  5  6  7 


2  3  4  5  6  7 

2  3  4  5  6  7 

2  3  4  5  6  7 


2  3  4  5  6  7 

2  3  4  5  6  7 


2  3  4  5  6  7 

2  3  4  5  6  7 

2  3  4  5  6  7 

2  3  4  5  6  7 

2  3  4  5  6  7 

2  3  4  5  6  7 
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Strongly 

Disagree 

Disagree 

Slightly 

Disagree 

Neutral 

Slightly 

Agree 

Agree 

Strongly 

Agree 

14. 

My  decision  making  ability  is  as  good  in  emergencies  as  it  is  in 
routine  mission  situations. 

1 

2 

3 

4 

5 

6 

7 

15. 

The  pilot-in-command  is  solely  responsible  for  leadership  of  the 
crew  team. 

1 

2 

3 

4 

5 

6 

7 

16. 

Pilots  should  consider  crew  chief  questions  and  suggestions. 

« 

2 

3 

4 

5 

6 

7 

17. 

When  joining  a  unit,  a  new  crewmember  should  not  offer 
suggestions  or  opinions  unless  asked. 

1 

2 

3 

4 

5 

6 

7 

18. 

Because  crew  chiefs  have  no  pilot  training,  they  should  limit 
their  attention  to  their  formally  defined  crew  chief  duties. 

1 

2 

3 

4 

5 

6 

7 

19. 

Pilots-in-command  who  accept  and  implement  suggestions  from 
the  crew  lessen  their  stature  and  reduce  their  authority. 

1 

2 

3 

4 

5 

6 

7 

20. 

Crewmembers  should  monitor  the  pilot-in-command’s 
performance  for  possible  mistakes  and  errors. 

1 

2 

3 

4 

5 

6 

7 

21. 

The  best  way  to  correct  an  error  is  to  alert  the  error  maker  so 
that  he  can  correct  the  problem. 

1 

2 

3 

4 

5 

6 

7 

22. 

Crewmembers’  errors  and  mistakes  during  the  mission, 
including  the  pilot-in-command’s  mistakes,  should  be  a 
significant  part  of  post  flight  crew  discussions. 

1 

2 

3 

4 

5 

6 

7 

23. 

The  pilot-in-command  should  seek  advice  from  crewmembers 
when  updating  mission  plans. 

1 

2 

3 

4 

5 

6 

7 

24. 

The  pilot-in-command  should  use  his  crew  to  help  him  maintain  1 
situation  awareness. 

2 

3 

4 

5 

6 

7 

25. 

The  pilot-in-command  is  solely  responsible  for  maintaining 
awareness  of  crew  capabilities. 

1 

2 

3 

4 

5 

6 

7 

26. 

Only  when  the  pilot-in-command  is  overloaded  should  he  pass 
workload  to  other  crewmembers. 

1 

2 

3 

4 

5 

6 

7 

27. 

Crewmembers  should  be  aware  of  other  crewmembers’ 
workload. 

1 

2 

3 

4 

5 

6 

7 

28. 

If  a  crewmember  is  having  difficulties  executing  his 
responsibilities,  other  crewmembers  should  provide  assistance. 

1 

2 

3 

4 

5 

6 

7 

29. 

Highly  competent  pilots  do  not  experience  task  overload. 

1 

2 

3 

4 

5 

6 

7 

30. 

A  crewmember  should  offer  task  help  to  another  crewmember 
only  if  he  is  sure  the  crewmember  needs  it. 

1 

2 

3 

4 

5 

6 

7 
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Strongly  Disagree  Slightly  Neutral  Slightly  Agree  Strongly 
Disagree  Disagree  Agree  Agree 

3 1 .  The  pilot-in-command  should  not  get  involved  with  the  1  2  3  4  5  6  7 

execution  of  responsibilities  assigned  to  other  crewmembers. 

32.  Crewmember  task  overload  usually  occurs  because  the  1  2  3  4  5  6  7 

crewmember  is  not  very  competent. 

33.  Pilots-in-command  should  employ  the  same  style  of  leadership  1  2  3  4  5  6  7 

in  all  situations  and  with  all  crewmembers. 

34.  Pilot-in>command  instructions  to  other  crewmembers  should  be  1  2  3  4  5  6  7 

general  and  non-specific  so  that  each  individual  can  practice 

self-management  and  can  develop  individual  skills. 

35.  A  relaxed  attitude  is  essential  for  maintaining  a  cooperative  and  1  2  3  4  5  6  7 

harmonious  cockpit. 

36.  Reprimands  are  more  efTective  than  discussions  in  eliminating  a  1  2  3  4  5  6  7 

crewmember’s  poor  flying  habit. 

37.  Nonrated  crewmembers  should  be  actively  involved  in  planning  1  2  3  4  5  6  7 

the  mission. 

38.  Understanding  the  commander’s  concept  is  of  minor  1  2  3  4  5  6  7 

importance  to  mission  execution. 

39.  Each  crewmember  should  watch  for  situations  in  which  1  2  3  4  5  6  7 

external  events  limit  others’  performance. 

40.  Thinking  through  difficult  segments,  events,  and  tasks  is  1  2  3  4  5  6  7 

primarily  the  pilot-in-command’s  responsibility. 

41.  My  knowledge  of  unit  SOP  and  aircraft  emergency  procedures  1  2  3  4  5  6  7 

makes  rehearsing  familiar  missions  unnecessary. 

42.  An  essential  element  of  premission  planning  is  discussing  crew  1  2  3  4  5  6  7 

responsibilities  and  required  actions  for  abnormal  events. 

43.  Recent  events  in  my  personal  life  have  little  to  do  with  my  1  2  3  4  5  6  7 

performance  as  a  crewmember. 

44.  Crewmembers  should  be  able  to  anticipate  requirements  as  the  1  2  3  4  5  6  7 

mission  progresses. 

45.  My  individual  performance  is  as  good  in  degraded  systems  I  2  3  4  5  6  7 

conditions  as  it  is  in  a  "full  up"  aircraft. 

46.  External  circumstances  require  crewmembers  to  provide  1  2  3  4  5  6  7 

situational  leadership  for  short  periods  of  time. 


Note:  The  Aircrew  Coordination  Evaluation  (ACE)  Checklist, 
developed  for  use  in  the  Field  Exportable  Evaluation  Package,  is 
presented  here  as  an  index  to  the  crew  coordination  Basic 
Qualities 


Rating  Scale 

The  following  numeric  rating  scale  is  used  to  assess  the  level  of 
behavior  that  crews  exhibit  for  each  basic  quality  shown  on  the 
Aircrew  Coordination  Evaluation  (ACE)  Checklist  (see  Figure  B-1) 
and  at  the  bottom  of  the  Aircrew  Coordination  Training  Grade 
Slip.  Each  basic  quality  is  rated  using  a  seven-point  scale  with 
values  ranging  from  1  (very  poor)  to  7  (superior) : 

Very  Accept-  Very 

Poor  Poor  Marginal  able  Good  Good  Superior 

1  2  3  4  5  5  7 

Rating  Guidelines 

Written  descriptions  of  the  types  of  behaviors  and  levels  of 
performance  are  shown  for  rating  values  1,  4,  and  7.  These 
descriptions  serve  as  behavioral  "anchors"  and  are  designed  to 
assist  evaluators  in  determining  how  well  a  crew  performs  on  each 
basic  quality  in  relation  to  a  well-defined  set  of  behaviors. 
Evaluators  should  use  the  "anchors"  as  the  standard  for  making 
ratings--avoid  comparing  one  crew's  performance  with  that  of 
another  crew's;  rate  a  crew's  performance  in  relation  to  the 
"anchors."  To  ensure  reliable  ratings,  continue  to  refer  to  the 
anchors  when  making  rating  responses  until  completely  confident 
and  understand  fully  how  to  rate  each  basic  quality. 

In  completing  a  basic  quality  rating,  evaluators  should  decide 
whether  the  behaviors  observed  fall  into  the  low  end  of  the  basic 
quality  range  (values  1  or  2),  the  middle  of  the  range  (values  3, 
4,  or  5),  or  the  high  end  of  the  range  (values  6  or  7).  Once  the 
general  range  of  response  is  selected,  use  the  anchors  to  help 
select  the  final  rating  value.  For  example,  if  a  crew  did  an 
adequate  job  of  pre-mission  planning  and  rehearsal,  the  rating 
would  come  from  the  middle  of  the  range  (3,  4,  or  5).  After 
determining  this,  review  the  behavioral  description  (anchor) 
associated  with  value  4  to  determine  if  crew  performance 
resembled  this  description  (4  value) ,  was  somewhat  less  than  this 
description  (3  value) ,  or  was  a  little  better  than  this 
description  (5  value) .  Use  the  end-point  anchors  similarly  to 
help  determine  ratings  that  fall  near  the  ends  of  the  scale. 

Army  aviation  crews  that  have  little  or  no  training  in  aircrew 
coordination  techniques  will  score  most  frequently  in  the  lower 
half  of  the  scale.  Most  other  crews,  however,  will  fall  into 
the  middle  area  of  the  scale.  Keep  in  mind  that  although  Amy 
aviators  have  well  developed  basic  flying  skills,  as  a  group, 
their  aircrew  coordination  skills  will  be  much  like  the 
rest  of  the  population.  A  few  crews  will  have  strong 
coordination  and  communication  skills,  a  few  will  have  weak 
skills,  and  a  significant  number  will  have  moderate  skills. 
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AIRCREW  COORDINATION  EVALUATION  (ACE)  CHECKUST 

For  use  of  this  form,  See  Aircrew  Coordination  Exportable  Evaluation 
Package  for  Army  Aviation. 


PC. 
PI  . 


Date 


NCM  . 


NO 


CREW  COORDINATION  BASIC  QUAUT1ES 


RATING 


Establish  and  maintain  flight  team  leadership  and  crew  climate 
(Crew  Climate) 


Premission  planning  and  rehearsal  accomplished  (Plan  Rehearse) 


Application  of  appropriate  decision  making  techniques  (Decision  Tech) 


Prioritize  actions  and  distribute  workload  (Workload) 


Management  of  unexpected  events  (Unexp  Events) 


Statements  and  directives  clear,  timely,  relevant  complete,  and  verified 
(Info  Xfer) 


Maintenance  of  mission  situationai  awareness  (Sit  Aware) 


Decisions  and  actions  communicated  and  acknowledged  (Comm/Ack) 


Supporting  information  and  actions  sought  from  crew  (Info  Sought) 


10 


Crewmember  actions  mutually  cross-monitored  (Cross  Monitor) 


11 


Supporting  information  and  actions  offered  by  crew  (Info  Offered) 


12 


Advocacy  and  assertion  practiced  (Advoc/Assert) 


13 


Crew-level  after-action  reviews  accomplished  (AAR) 


Evaluator's  Signature: 


Notes: 

Consult  the  behavioral  anchored  rating  guidance.  Enter  a  summary  rating  (1, 2  ...  7)  in  the  rating 
btock  for  each  Basic  Quality.  Refer  to  the  rating  scale  below. 


RATING  SCALE 


Very  Poor 

Poor 

Marginal 

Acceptable 

Good 

Very  Good 

1 

2 

3 

4 

5 

6 

Superior 

7 


AIRCREW  COORDINATION  EVALUATION  (ACE)  CHECKUST 


Figure  B-1.  Behavioral  anchored  ratings 


Aircrew  Coordination  Basic  Qualities  and  Behavioral  Anchors 


BASIC  QUALITY  1.  EstabllBh  and  maintain  flight  team  leadership 

and  crew  climate  (Crew  Climate) 


Explanation: 

This  rating  assesses  the  quality  of  relationships  among  the  crew 
and  the  overall  climate  of  the  flight  deck.  Aircrews  are  teams 
with  a  designated  leader  and  clear  lines  of  authority  and 
responsibility.  The  pilot-in-command  sets  the  tone  of  the  crew 
and  maintains  the  working  environment.  Effective  leaders  use 
their  authority  but  do  not  operate  without  the  participation  of 
other  crewmembers.  When  crewmembers  disagree  on  a  course  of 
action,  rate  the  crew's  effectiveness  in  resolving  the 
disagreement.  Note:  Traditional  leadership  centralizes 
leadership  in  the  leader  with  followers  fully  dependent  on  the 
leader.  Functional  leadership  assigns  leadership  and 
followership  roles  as  the  situation  evolves.  Flight  team 
leadership  recognizes  the  impact  of  leadership  style  on  the 
working  environment.  Regardless  of  leadership  style,  the  pilot- 
in-command  retains  final  decision  and  direction  authority. 

Superior  Rating  (7) 

The  crewmembers  have  very  good  interpersonal  relationships.  They 
respect  each  others’  skills  and  appear  to  enjoy  being  with  each 
other.  The  climate  is  very  open;  crewmembers  freely  talk  and  ask 
questions.  Crewmembers  encourage  the  individual  with  the  most 
information  about  the  situation-at-hand  to  participate.  There  is 
a  genuine  concern  for  good  working  relationships.  No  degrading 
comments  or  negative  voice  tones  are  used  in  interactions . 
Disagreements  are  perceived  as  a  normal  part  of  crew 
interactions,  and  the  crew  directly  confronts  the  issues  over 
which  the  disagreement  began.  Arguments  or  disagreements  focus 
on  behaviors  or  solutions  rather  than  on  personalities.  Each 
crewmember  carefully  listens  to  others'  comments.  Senior 
crewmembers  accept  challenges  from  junior  crewmembers. 

Alternative  solutions  are  explored.  The  solution  produced  is  a 
"win-win"  situation  in  which  all  crewmembers’  opinions  are 
considered.  The  crewmembers  have  no  hard  feelings  at  the 
conclusion  of  the  incident. 


Accepteible  Rating  (4) 

The  crewmembers  have  sound  interpersonal  relationships  and  seem 
to  respect  each  others’  skills.  The  climate  is  an  open  one,  and 
crewmembers  are  free  to  talk  and  ask  mission  questions. 

Regardless  of  rank  or  duty  position,  the  individual  with  the  most 
information  about  the  situation-at-hand  is  allowed  to 
participate.  When  disagreements  arise, -'the  crew  directly 
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confronts  the  issues  over  which  the  disagreements  began.  The 
primary  focus  is  on  beha^viors  or  solutions,  and  no  personal 
attacks  are  made  in  the  heat  of  discussion.  The  solution  is 
generally  seen  as  reasonable.  Problem  resolution  ends  on  a 
positive  note  with  very  little  hostility  or  grumbling  among 
crewmembers.  Mutual  respect  is  clearly  intact. 


Very  Poor  Rating  (1) 

Crew  interactions  are  often  awkward  and  uncomfortable.  The 
crewmembers  do  not  appear  to  like  or  respect  each  other. 
Crewmembers  m.ay  be  curt  and  impolite  to  each  other.  Requirements 
for  assistance  are  made  as  commands  rather  than  as  requests  for 
support.  When  disagreements  arise,  the  crew  fails  to  directly 
confront  the  issues.  Personal  attacks  may  arise.  Senior 
crewmembers  are  resistant  to  recommendations  from  junior 
crewmembers.  Crewmembers  do  not  explore  the  range  of  possible 
solutions.  They  may  shout  and  argue  without  finding  a  solution. 
One  or  more  crewmembers  may  retreat  and  say  nothing  at  all .  A 
"win-lose"  situation  develops  in  which  one  crewmember  is  shown  to 
be  right  and  the  other  to  be  wrong.  The  crewmembers  show  little 
respect  to  one  another  except  for  deferring  to  formal  rank. 


BASIC  QUALITY  2.  Pre-ml3S±on  s>lazmlnsr  and  rehearsal 

accongpllshed  (Plan  Rehearsal) 


Explanation: 

This  rating  assesses  the  pre-mission  planning  and  rehearsal 
activities  that  the  crew  performs  upon  receiving  a  mission  order. 
Time  available  determines  whether  pre-mission  planning  and 
rehearsal  is  completed  prior  to  the  flight  or  in  the  cockpit. 
During  this  period  crews -- 

•Clarify  the  mission  order  and  the  commander's  intent 
•Assign  actions,  duties,  and  mission  responsibilities 
•Collect  information  (intelligence,  communications, 
weather,  flight  planning)  and  develop  the  plan 
•Conduct  crew  briefing  to  review  and  discuss  the  plan 
•Identify  potential  problem  areas  and  courses  of  action 
•Assess  risks 

•Visualize  and  rehearse  the  mission 

Although  the  pilot-in-command  is  responsible  for  leading  this 
activity,  evaluate  the  extent  and  manner  in  which  the  entire  crew 
participates.  Also,  consider  the  time  constraints  on  the  crew.  If 
there  is  insufficient  time  to  conduct  comprehensive  planning  and 
rehearsal,  evaluate  the  crew  on  their  planning  and  rehearsal  of 
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the  most  critical  segments  of  the  mdssion.  That  is,  either  prior 
to  the  flight  or  in  the  cockpit,  did  the  crew  address  the  most 
important  issues  given  the  time  available?  Note:  The 
relationship  among  crew  members  should  be  observed  during  this 
period  but  the  crew  climate  evaluation  should  be  made  on  rating 
basic  quality  1,  Flight  Team  Leadership  and  Crew  Climate. 

Exampl es : 

•UH-60  Task  2078  and  AH-64  Task  1033,  Perform  terrain  flight 
mission  planning:  The  crew  will  analyze  the  mission  in  terms  of 
METT-T  and  plan  the  flight  as  directed  by  the  PC.  The  crew  will 
rehearse  important  aspects  of  the  mission. 

•UH-60  and  AH-64  Task  1000,  Conduct  crew  mission  briefing: 

Aircrew  collectively  visualizes  and  rehearses  expected  and 
unexpected  events  from  takeoff  to  tie-down;  all  factors  of  the 
flight;  and  actions,  duties,  and  responsibilities  of  each 
crewmember . 

•AH-64  and  UH-60  Task  1068,  Perform  or  describe  emergency 
procedures:  PC  will  include  in  the  crew  briefing  the  general 
approach  to  all  emergency  procedures  requiring  immediate  action. 

Superior  Rating  (7) 

The  entire  crew  discusses  a  detailed  description  of  the  mission 
and  the  commander's  intent.  All  actions,  duties,  and  mission 
responsibilities  are  partitioned  and  clearly  assigned  to  specific 
individuals.  The  crew  acquires  new  and  updated  information  and 
uses  it  to  develop  the  mission  plan  from  the  aircrew  mission 
briefing.  Questions  and  discussion  about  the  mission,  commander's 
intent,  and  specific  responsibilities  are  encouraged.  Potential 
problems  are  noted  and  discussed  in  detail .. Courses  of  action  and 
individual  responsibilities  are  established  in  the  event  that 
potential  problems  actually  occur.  All  crewmembers  speak  out  and 
acknowledge  an  understanding  of  the  operational  risks  in  the 
mission  plan.  The  pi lot -in-command  leads  the  crew  in  mentally 
rehearsing  the  entire  mission  by  visualizing  and  talking  the  crew 
through  potential  problems  and  contingencies.  Crewmembers 
acknowledge  understanding  their  assigned  responsibilities  and 
cues  for  actions.  The  tone  of  the  interaction  is  friendly  and 
professional . 


Accept2ible  Rating  (4) 

A  brief  description  of  the  mission  is  provided  to  the  entire 
crew.  The  mission  responsibilities  are  partitioned  and  assigned 
to  specific  individuals.  Actions  are  taken  to  update  current 
information  that  adds  to  the  aircrew  mission  briefing  and  helps 
develop  the  mission  plan.  One  or  more  crewmembers  make  comments 
during  the  course  of  developing  the  mission  plan.  Potential 
mission  problems  are  only  briefly  discussed.  There  is  adequate 
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preparation  for  contingencies.  Crewmembers  briefly  discuss  the 
operational  risks  in  the.  mission  plan.  Mental  rehearsal  is 
initiated  by  the  pilot-fn-command  or  another  crewmember  who  talks 
through  potential  problems  or  contingencies  for  one  or  more 
mission  segments.  Some  discussion  takes  place  to  clarify 
responsibilities  in  the  event  of  unexpected  problems  or 
contingencies.  The  tone  of  the  interaction  is  generally  friendly 
and  businesslike. 


Very  Poor  Rating  (1) 

The  pilot-in-command  briefs  the  mission  with  little  or  no 
attendant  explanation.  There  is  little  or  no  discussion  of 
responsibilities  or  their  assignments  to  specific  crewmembers. 

The  pilot -in-command  develops  the  mission  plan  from  the  aircrew 
mission  briefing  and  current  information.  Crewmembers  tend  not  to 
ask  questions  about  the  mission.  If  asked,  questions  tend  to  be 
cut  off,  only  briefly  addressed,  or  ignored  by  the  other  crew¬ 
members.  Little  or  no  mention  is  given  to  potential  problems  or 
complications.  No  crewmember  says  anything  about  operational 
risks  or  weaknesses  in  the  plan.  Any  suggestion  to  talk  through  a 
potential  problem  or  mentally  rehearse  responsibilities  is 
rejected  as  unnecessary.  The  tone  of  the  interaction  is 
business-like,  abrupt,  and  impersonal. 


BASIC  QUALITY  3,  Application  of  appropriate  deal -making 

technignes  (Decision  Tech) 


Explanation: 

This  rating  evaluates  the  manner  and  quality  of  the  crew's 
problem  solving  and  decision  making  performance  throughout  the 
planning  and  execution  of  the  mission.  Factors  to  consider  in 
making  this  evaluation  include  (1)  information  available  to  the 
crewmembers,  (2)  time  urgency  of  the  decision,  (3)  objectivity 
reflected  in  the  decision  process,  and  (4)  level  of  involvement 
and  information  exchange  among  the  crewmembers.  The  time 
critical  demands  of  tactical  flying  require  many  decisions  to  be 
made  on  an  automatic,  pattern-recognition  basis  with  only  a 
minimum  level  of  information  exchange.  However,  when  adequate 
time  and  information  are  available,  crewmembers  are  expected  to 
engage  in  a  more  deliberate  and  interactive  style  of  decision 
making.  The  evaluation  of  crew  decision  making  performance 
should  ask  the  following  questions:  (1)  Did  the  crew  use  all  of 
the  available  information?  (2)  Was  the  level  of  information 
exchange  among  crewmembers  appropriate  for  the  time  available? 
(3)  Was  the  type  of  decision  process  (deliberate  versus 
automatic)  appropriate  for  the  time  available? 
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Examples: 

•UH-60  and  AH-64  Task  2044,  Perform  actions  on  contact:  Crew  will 
discuss  options  for  developing  the  situation,  then  choose  a 
course  of  action  that  supports  the  intent  of  the  unit  commander's 
directives . 

•AH-64  and  UH-60  Task  2083,  Negotiate  wire  obstacles:  Crew  will 
discuss  the  characteristics  of  the  wires  ...  to  determine  the 
method  of  crossing. 

Superior  Rat ing  ( 7 ) 

Crew  decision  making  consistently  reflects  proper  attention  to 
available  information  throughout  mission  planning  and  execution. 
The  level  of  crew  participation  and  deliberate  analysis  of 
options  is  appropriate  for  the  decision  time  available.  Resulting 
decisions  are  timely  and  appropriate  given  the  time  urgency  and 
level  of  information  available  in  each  situation.  Crewmembers  do 
not  exhibit  any  of  the  known  hazardous  thought  patterns  (e.g., 
anti-authority,  impulsivity,  machoism,  invulnerability, 
resignation,  get-home-itis,  overconfidence  in  other  aviator)  and 
appear  motivated  to  seek  the  most  mission  effective  and  safe 
decision  in  each  situation.  The  crew  decides  and  implements  a 
course  of  action  before  the  situation  jeopardizes  crew 
performance  or  mission  accomplishment. 


AcceptcUsle  Rating  (4) 

Crew  decisions  occasionally  reflect  inadequate  sharing  or  use  of 
available  information.  On  limited  occasions,  crewmembers  dwell 
excessively  on  some  issues  while  neglecting  more  time  urgent 
requirements.  Most  decisions  are  timely,  but  crew  perfomance 
begins  to  show  signs  of  self-induced  stress.  Most  decisions  are 
appropriate  for  the  situation,  with  the  crew  occasionally 
overlooking  one  or  more  factors  or  options.  Crewmembers 
occasionally  fail  to  recognize  or  exploit  opportunities  for 
additional  planning  or  rehearsal,  substituting  instead  ad  hoc 
strategies  or  plans.  Crewmembers  do  not  exhibit  any  of  the  known 
hazardous  thought  patterns.  The  situation  may  worsen,  without 
seriously  degrading  mission  accomplishment,  before  the  crew 
decides  and  implements  a  course  of  action. 


Very  Poor  Rating  (1) 

Crew  performance  (both  pre-flight  and  in-flight)  reflects  an 
inflexible  style  of  decision  making  (either  deliberate  or 
automatic)  regardless  of  time  urgency.  Crewmembers  may  engage  in 
excessive  deliberation,  overlook  the  relative  time  urgency  of 
competing  decision  requirements,  or  fall  victim  to  inappropriate 
mind  sets.  As  a  result,  decisions  frequently  lack  timeliness, 
ignore  important  factors,  or  appear  out  'of  context.  Information 
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exchange  and  crewmember  interaction  is  minimal,  with  the  result 
that  critical  input  is  ignored  or  not  sought.  Crewmembers  may 
display  one  or  more  of  Che  known  hazardous  thought  patterns 
(e.g.,  machoism,  anti-authority,  get-home-itis) .  The  crew  may  be 
unable  to  decide  or  implement  a  course  of  action  before  a 
situation  becomes  critical. 


BASIC  QUALITY  4,  Prioritize  actions  and  distribute  workload 

(Workload) 


Explanation: 

This  is  a  rating  of  the  effectiveness  of  time  and  work 
management.  Rate  the  extent  to  which  the  crew  as  a  team  avoids 
being  distracted  from  essential  activities,  distributes  workload, 
and  avoids  individual  crewmember  overload. 

Examples: 

♦AH-64  and  UH-60  Task  1080,  Perform  procedures  for  two-way  radio 
failure:  P*  will  remain  focused  outside  the  aircraft  or  inside 
the  cockpit  on  the  instruments,  as  appropriate.  He  will  not 
participate  in  troubleshooting  the  malfunction. 

♦UH-60  Task  2079  and  AH-64  Task  1064,  Perform  terrain  flight 
navigation:  P  will  focus  his  attention  primarily  inside  the 
cockpit;  however,  as  workload  permits,  he  will  assist  in  clearing 
the  aircraft  and  provide  adequate  warning  of  traffic  and 
obstacles . 


Superior  Rating  (7) 

Virtually  all  distractions  are  avoided.  Each  crewmember 
understands  precisely  what  information  is  relevant  to  the  mission 
and  what  information  is  simply  a  distraction.  If  a  crewmember 
becomes  mildly  distracted,  other  crewmembers  remind  him  to  focus 
on  Che  mission  Cask.  Non-critical  duties  are  prioritized  and 
delayed  until  low  workload  periods  or  post-flight  periods. 
Crewmembers  are  aware  of  workload  build  ups  on  others  and 
readjust  workload  by  assuming  emerging,  unassigned  tasks 
appropriate  for  their  duty  station.  Overloads  do  not  occur.  The 
crew's  planning  horizon  is  always  "ahead  of  the  aircraft." 


Acceptable  Rating  (4) 

Most  distractions  are  avoided.  The  crew  performs  well  in 
deciding  what  information  and  activities  are  essential  to  the 
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mission.  Most  non-essential  information  is  discarded  or  ignored. 
Non-critical  duties  are  prioritized  and  delayed  until  low 
workload  periods  or  post-flight  periods.  Crewmembers  are  aware 
of  individual  crewmember  workloads  during  each  phase  of  the 
mission.  When  an  individual  crewmember  appears  to  be  overloaded, 
other  crewmembers  take  on  part  of  the  workload.  The  crew  is 
always  "in  sync  with  the  aircraft." 


Very  Poor  Rating  (1) 

The  crew  is  easily  distracted.  The  crew  is  unable  or  unwilling 
to  decide  what  is  important  and  relevant  to  the  immediate 
mission.  There  is  little  prioritizing  of  duties  or  actions. 

Time  and  energy  may  be  wasted  on  low  priority  tasks.  Risks  to 
crew  safety  may  occur  as  the  crew  focuses  on  minor  tasks  while 
critical  tasks  requiring  immediate  attention  go  unattended, 

(e.g.,  setting  a  radio  frequency  when  attention  should  be  focused 
on  clearing  an  obstacle.).  Neither  the  overloaded  party  nor 
other  crewmembers  takes  voluntary  actions  to  eliminate  an 
overload  condition.  The  crew  makes  little  or  no  effort  to 
redistribute  task  responsibilities  as  mission  changes  occur  and 
new  tasks  arise.  Individual  crewmembers  experience  workload 
overloads.  The  crew's  planning  horizon  is  sometimes  "behind  the 
aircraft . “ 


BASIC  QUALITY  5.  Management  of  unexpected  events  (Unexp  Events) 


Explanation: 

This  rating  evaluates  the  crew's  performance  under  unusual 
circumstances  that  may  involve  high  levels  of  stress.  This 
judgement  includes  the  integration  of  technical  and  managerial 
aspects  of  contending  with  the  situation.  Note:  Enter  the 
abnormal  or  emergency  situation  in  the  Aircrew  Coordination 
Training  Grade  Slip  (some  emergency  procedure  ATM  tasks  are 
preprinted)  and  grade  it  the  same  as  any  task. 

Exampl es : 

•AH-64  and  UH-60  Task  2008,  Perform  evasive  maneuvers:  The  most 
important  consideration  in  an  emergency  is  aircraft  control-- 
first  assess  aircraft  controllability,  check  systems  indicators, 
take  evasive  action. 

•UH-60  Task  1068,  Perform  or  describe  emergency  procedures:  CE 
will  keep  communications  to  a  minimum  to  allow  the  P*  or  P  to 
attempt  communications  outside  the  aircraft. 
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Superior  Rating  (7) 

The  crew  remains  calm  during  the  situation.  Each  crewmember 
seeks  to  understand  the  problem  and  provides  the  pilot-in-command 
with  essential  information.  Each  crewmember  immediately  takes  on 
particular  workload  responsibilities  based  on  prior  discussions 
and  rehearsal  of  potential  problems  and  contingencies.  The  crew 
effectively  communicates  its  actions  and  results  to  others  and 
provides  feedback  to  ensure  complete  coordination  of  efforts. 

Each  crewmember  handles  his  own  responsibilities  and  seeks  to 
support  the  crewmember  with  the  greatest  workload.  The  crew 
rapidly  imposes  the  maximum  amount  of  control  possible  over  the 
situation  given  the  available  time  and  internal  and  external 
resources.  A  high  level  of  situation  awareness  is  maintained 
throughout  the  event . 


Acceptcible  Rating  (4) 

The  crew  responds  to  the  problem  and  the  pi lot- in-command ' s 
requests  for  information  but  does  not  overreact..  The 
pilot-in-command's  requests  for  information  are  met  by  feedback 
from  the  crew.  The  crew  takes  actions  to  reduce  the 
pilot-in-command’s  work  overload  and  provides  information  even  if 
it  is  not  specifically  requested.  The  pilot  and  crew  make  good 
use  of  available  resources.  The  crew  is  intense  but  not 
flustered  by  the  situation.  Adequate  situation  awareness  is 
maintained  throughout  the  event . 


Very  Poor  Rating  (1) 

The  crew  becomes  disorganized  and  flustered.  The  pilot-in¬ 
command’s  requests  for  information  elicit  inadequate  responses. 
Crewmembers  may  focus  on  the  wrong  issues,  thus  delaying  correct 
diagnosis  of  the  problem.  The  crew  focuses  on  only  one  solution 
to  an  event,  does  not  consider  other  plausible  alternatives,  or 
chooses  an  inappropriate  solution.  Lack  of  coordinated  actions 
adds  to  the  confusion.  The  pilot  and  crewmembers  make  poor  use 
of  available  resources  to  resolve  the  problem.  Situation 
awareness  appears  to  decay  during  the  situation. 


BASIC  QUALITY  6.  Statements  and  directives  clear,  timely, 

relevant,  coiaplete,  and  verified  (Info  Xfer) 


Explanation: 

Rate  the  completeness,  timeliness,  and  quality  of  information 
transfer.  Carefully  consider  the  crew’s  feedback  techniques  to 
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verify  information  transfer.  In  particular,  evaluate  the  quality 
of  instructions  and  statements  associated  with  navigation 
activities,  obstacle  clearing  activities,  and  instrument 
readouts . 

Exampl es : 

•AH-64  Task  1015,  Perform  ground  taxi:  The  P  will  announce 
"Blocking"  to  acknowledge  the  P* ' s  announcement  "Braking". 

•UH-60  Task  2079,  Perform  terrain  flight  navigation:  The  P*  will 
acknowledge  commands  issued  by  the  P  for  heading  and  airspeed 
changes . 

Superior  Rating  (7) 

Crewmembers  communicate  frequently.  Both  senders  and  receivers 
use  standard  terminology  for  nearly  all  communications.  Senders 
almost  always  provide  clear,  concise  information.  Receivers 
acknowledge  nearly  all  messages  in  sufficient  detail  so  that  the 
sender  can  verify  that  the  receiver  understands  the  message. 
Receivers  ask  for  clarification  when  they  do  not  understand. 
Senders  pursue  feedback  when  no  response  is  forthcoming.  Whenever 
a  workload  shift  or  task  responsibility  transfer  occurs,  the 
change  is  communicated  and  acknowledged  by  the  crew.  All 
navigation,  obstacle  clearing,  and  "inside"  or  "outside"  the 
cockpit  information  is  stated,  acknowledged,  and  updated. 


Acceptable  Rating  (4) 

Crewmembers  communicate  about  the  mission  as  required.  Standard 
terminology  is  usually  used.  Receivers  acknowledge  most 
messages.  Receivers  ask  questions  when  they  do  not  understand. 
Senders  usually  pursue  feedback  when  no  response  is  forthcoming. 
Crewmembers  are  appraised  of  changes  to  responsibilities  during 
the  flight.  "Inside"  and  "outside"  the  cockpit  duties  are 
specified  and  communicated  to  others. 


Very  Poor  Rating  (1) 

Crewmembers  may  fail  to  make  statements  regarding  critical 
information.  Non-standard  terminology  is  used  or  standard 
terminology  is  used  inappropriately.  Sender  messages  may  be 
inappropriately  delayed  or  irregular  and  may  be  confusing. 
Receivers  usually  do  not  verbally  acknowledge  the  receipt  of 
messages.  Receivers  do  not  ask  questions.  Senders  do  not  pursue 
feedback  when  no  response  is  forthcoming.  Changes  in 
responsibilities  during  the  mission  are  often  not  communicated 
and  may  result  in  confusion  over  who  has  a  task  responsibility. 
Navigation  instructions  and  obstacle  location  information  may  be 
incomplete  or  confusing.  At  times,  "inside"  or  "outside"  the 
cockpit  responsibilities  are  not  clearly*  communicated. 
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BASIC  QUALITY  7.  Maintenance  of  mlsBlon  situation  awareness 

(Sit  Aware) 


Explanation: 

This  rating  assesses  the  extent  to  which  crewmembers  keep  each 
other  informed  on  the  status  of  the  aircraft  and  mission 
accomplishment.  This  information  reporting  helps  maintain  a  high 
level  of  situation  awareness  among  the  flight  crew.  Information 
reported  includes : 

♦Aircraft  position  and  orientation 
♦Equipment  status 
♦Personnel  status 

♦Environment  and  battlefield  conditions 
♦Changes  to  mission  objectives 

Crew-wide  situation  awareness  is  an  essential  element  of  safe 
flying  and  effective  crew  performance. 

Examples: 

♦UH-60  Task  2009,  Perform  multi-aircraft  operations:  P  and  CE 
will  provide  adequate  warning  to  avoid  traffic  or  obstacles. 

♦AH-64  Task  2008,  Perform  evasive  maneuvers:  When  engaged  by  the 
enemy,  crew  will  announce  the  nature  and  direction  of  the  threat. 

Superior  Rating  (7) 

Crewmembers  routinely  provide  each  other  with  updates  on  the 
status  of  the  elements  of  situation  awareness  and  the  status  of 
the  mission.  Crewmembers  anticipate  the  situation  awareness 
needs  of  others  and  request  needed  information  when  it  is  not 
forthcoming.  Crewmembers  are  aware  of  each  others’  mental  and 
physical  states  and  are  not  hesitant  to  alert  others  to  personal 
problems  that  could  undermine  effective  performance.  Personnel 
status  is  voluntarily  shared  without  fear  of  sanctions.  All 
changes  in  the  elements  of  situation  awareness  are  verbalized  and 
acknowledged.  Crewmembers  alert  other  crewmembers  to  the 
presence  of  obstacles. 


AcceptaJole  Rating  (4) 

Crewmembers  usually  provide  updates  on  the  status  of  most  of  the 
elements  of  situation  awareness  and  the  status  of  the  mission. 
Changes  to  the  situation  awareness  elements  are  verbalized. 
Obvious  changes  in  personnel  status  are  noted  and  acknowledged 
without  fear  of  sanctions. 
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Very  Poor  Rating  (1) 


Crewmembers  do  not  routinely  provide  updates  on  the  status  of  the 
aircraft  or  the  status  of  the  mission.  Generally,  updates  are 
provided  only  on  request;  they  are  not  made  voluntarily. 

Personnel  problems  such  as  fatigue  or  lack  of  attention  are  not 
mentioned. 


BASIC  QUALITY  8.  Decisions  and  actions  communicated  and 

acknowledged  ( Comm/  Ack) 


Explanation: 

C-ate  the  extent  to  which  decisions  and  actions  are  actually  made 
and  announced  to  the  crewmembers  after  input  is  solicited  from 
them.  Crewmembers  should  respond  verbally  or  with  the 
appropriate  adjustment  to  their  behaviors,  actions,  or  control 
inputs  to  clearly  indicate  that  they  understand  when  a  decision 
has  been  made  and  what  it  is.  Failure  to  do  so  may  confuse  crews 
and  lead  to  uncoordinated  operation.  Note:  Due  to  time 
constraints  in  certain  situations,  there  is  often  little  or  no 
time  for  crews  to  make  inputs  to  a  decision.  In  such  cases, 
raters  should  focus  on  the  extent  to  which  decisions  are 
acknowledged  verbally  or  through  coordinated,  pre-planned  action. 

Examp 1 es ; 

•UH-60  Task  2086,  Perform  masking  and  unmasking:  P*  will  announce 
his  intent  to  unmask.  The  P  and  CE  will  acknowledge  that  they 
are  prepared  to  execute  the  maneuver. 

•AH-64  Task  1038,  Perform  terrain  flight  approach:  P*  will _ 
announce  intention  of  a  go-around  .  .  .  whether  approach  will 
terminate  to  a  hover  or  to  the  ground.  P  will  acknowledge  use  of 
manual  stabilator  or  any  intent  to  deviate  from  the  approach. 

Superior  Rating  (7) 

The  pi lot -in-command  states  decisions  and  actions  and,  time 
permitting,  explains  the  reasons  and  intent.  Crewmembers 
acknowledge  the  decisions  with  a  clear  verbal  response  and  ask 
questions  to  clarify  any  confusion.  The  pilot-in-command  answers 
all  questions  in  a  positive,  straight-forward  manner. 

Crewmembers  keep  the  pilot-in-command  informed  of  the  results  of 
their  activities  and  changing  responsibilities--especially  visual 
area  of  responsibility  or  task  focus.  The  crew  clearly 
acknowledges  results  of  actions,  or  changes,  and  then  states  its 
intended  adjustments  based  on  the  information  provided.  If 
crewmembers  do  not  acknowledge  or  adjust^  the  pilot-in-command 
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requests  acknowledgement.  Crewmembers  are  particularly  attentive 
to  the  communication  of  workload  responsibilities.  When  assuming 
control  of  the  aircraft'br  making  control  inputs,  notification  is 
always  given  and  acknowledgement  received. 


Acceptable  Rating  (4) 

The  pilot-in-command  states  decisions  and  actions  along  with, 
time  permitting,  a  brief  explanation  of  the  reasons  and  informs 
the  crew  of  the  adjustments  they  are  expected  to  make.  The  crew 
acknowledges  its  awareness  of  the  decisions  and  directions. 
Crewmembers  may  ask  questions  to  clarify  confusion.  The  pilot 
answers  questions  clearly  and  quickly  and  the  crew  adjusts  to  the 
new  situation.  When  assuming  control  of  the  aircraft  or  making 
control  inputs,  notification  is  given  and  acknowledged. 


Very  Poor  Rating  (1) 

Decisions  and  actions  of  a  crewmember  are  often  not  passed  on  to 
the  crew.  The  pilot-in-command  takes  unilateral  action  and  does 
not  explain  or  inform  the  crew  of  his  intended  purpose.  The  crew 
is  often  not  aware  that  a  decision  has  been  made.  The  crew 
infrequently  asks  questions  for  clarification.  The 
pilot-in-command  may  not  acknowledge  or  respond  to  questions.  The 
crew  may  not  know  how  to  react  to  changed  circumstances. 
Crewmembers  are  often  unsure  what  responsibilities  have  been 
assigned  to  them.  Crewmembers  may  take  uncoordinated  actions 
without  stating  intentions  or  results.  Two  pilots  may  attempt  to 
simultaneously  take  control  of  the  aircraft  when  flight  control 
authority  is  unclear. 


BASIC  QUALITY  9.  Supporting  Information  and  actions  sought  from 

crew  (Info  Sought) 


Explanation: 

This  is  a  rating  of  the  extent  to  which  crewmembers,  usually  the 
pilot -in-command,  seek  support  information  and  support  actions 
from  the  crew.  Evaluate  the  degree  to  which  crewmembers  raise 
questions  during  the  flight  regarding  plans,  revisions  to  plans, 
actions  to  be  taken,  and  the  status  of  key  mission  information. 
Noce:  The  extent  to  which  crewmembers  maintain  situational 
awareness  and  contribute  to  decision  making  should  be  observed 
here  but  evaluated  on  basic  qualities  7  and  4  respectively. 

Examp  1 es : 


B-14 


4 


•UH-60  Task  1032,  Perform  slope  operations:  P*  will  request 
assistance  in  setting  the  brakes. 

•AH-64  Task  2044,  Perform  actions  on  contact:  The  crew  will 
discuss  options  for  developing  the  situation. 


Superior  Rating  (7) 

During  the  flight,  crewmembers  raise  questions  on  plans  or 
changes  to  plans  and  actions.  Virtually  all  of  these  inquiries 
surface  information  that  contributes  to  the  mission  decision 
making  process.  When  the  pi lot -in-command  realizes  that  a 
decision  must  be  made  during  the  flight,  for  which  there  is  no 
clear  standardized  answer,  he  immediately  alerts  the  crew  to  the 
situation  and  seeks  suggestions  on  possible  solutions  and 
important  information  to  consider.  The  pilot-in-command  is  open 
to  all  suggestions.  Crewmembers  respond  to  these  inquiries  with 
sound,  task-focused  discussions  and  clear  answers  that  are 
provided  in  a  timely  manner.  Crewmembers'  inquiries  are  never 
ignored.  All  crewmembers  encourage  such  questioning.  When  the 
pilot-in-command  asks  for  assistance  with  actions  he  clearly 
states  what  assistance  is  required.  He  provides  quick,  clear 
feedback  if  the  crewmember  response  is  not  what  he  expects.  He 
asks  for  assistance  before  becoming  overloaded. 


Acceptable  Rating  (4) 

During  the  flight,  crewmembers  occasionally  raise  questions  on 
plans  or  actions  when  they  are  unclear  on  decisions  being  made. 
Most  of  these  inquiries  provide  information  that  is  relevant  to 
the  mission  decision  making  process.  The  pilot  alerts  the  crew 
to  the  need  for  decision  input.  Crewmembers  usually  respond  to 
these  inquiries  with  brief  but  reasonable  answers.  Crewmembers' 
inquiries  are  encouraged  by  other  crewmembers  most  of  the  time. 
The  pilot-in-command  listens  to  suggestions  without  interruption 
or  criticism.  He  asks  for  clarification  as  necessary.  He  only 
asks  for  assistance  when  he  becomes  overloaded. 


Very  Poor  Rating  (1) 

During  the  flight,  crewmembers  almost  never  raise  questions  about 
plans,  actions,  or  changes  to  plans.  The  pilot -in-command  makes 
mission  decisions  without  seeking  inputs  from  other  crewmembers. 
The  pi lot -in-command  does  not  alert  the  crew  that  a  decision  is 
required  or  is  being  made.  Decision  making  and  planning  are  done 
by  one  individual  with  little  or  no  discussion--an  observer  will 
have  difficulty  noting  this  quality  for  "very  poor"  crews  since 
it  is  hard  to  detect  individual  decision  making.  The  few 
inquiries  that  are  made  are  generally  ignored  or  abruptly 
answered.  Crewmembers  may  discourage  others  from  asking 
questions  by  the  tone  of  voice  they  use  or  by  failing  to  respond. 
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The  pilot-in-command  may  not  ask  for  crew  assistance  with  tasks 
even  when  he  is  overloaded  to  the  point  of  nearly  failing  to 
properly  execute  tasks. 


BASIC  QUALITY  10.  Crewmember  actions  mutually  cross  monitored 

(Cross  Monitor) 


Explanation: 

This  rating  captures  the  extent  to  which  a  crew  uses  cross 
monitoring  as  a  mechanism  to  avoid  errors  and  improve  future 
performance.  Crewmembers  are  able  to  catch  each  other's  errors. 
Such  redundancy  is  particularly  important  when  crews  are  fatigued 
or  overly  focused  on  critical  task  elements,  and  thus  more  prone 
to  make  errors.  Included  in  this  rating  is  the  crew's  use  of 
aircraft  technical  manual  checklists  to  perform  required 
procedure  checks  and  procedures  (i.e.,  engine-start,  run-up, 
before-takeoff,  before-  and  after-landing,  shutdown  checks;  HIT 
and  emergency  procedures).  Note:  This  quality  does  not  imply 
that  task  responsibilities  are  not  clearly  defined.  It  asks  the 
question  "To  what  extent  do  crewmembers  help  an  individual 
assigned  primary  responsibility  for  a  task  or  action  by  reviewing 
the  quality  of  that  individual's  task  execution  and  alerting  him 
to  any  mistake  noted?" 

Examp  1 es : 

•AH-64  Task  1094,  Identify  major  US  or  allied  equipment  and  major 
threat  equipment:  P*  or  P  will  announce  the  type  and  direction  of 
the  equipment  detected.  The  other  crewmember  will  confirm  the 
type  and  direction  of  the  equipment. 

♦UH-60  task  1023,  Perform  fuel  management  procedures:  PC  will 
confirm  the  results  of  the  fuel  check. 

Superior  Rating  (7) 

Each  crewmember  is  concerned  that  all  tasks  are  properly  executed 
and  checks  both  his  tasks  and  those  of  others .  When  mistakes  are 
noted,  the  crewmember  making  the  error  is  quickly _ informed  in  a 
concise  manner  without  excessive  formality.  The  mistake  maker 
accepts  this  review  and  feedback  as  a  normal  part  of  crew 
operations . 


AcceptcdDle  Rating  (4) 

Crewmembers  often  check  each  other's  task  performance  for  errors. 
Mistake  makers  are  informed  and  make  the  needed  corrections.  Only 
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occasionally  are  mistake  makers  annoyed  at  being  checked  and 
corrected. 


Very  Poor  Rating  (1) 

Crewmembers  seldom,  if  ever,  check  each  other's  task  execution. 
Crewmembers  are  insulted  if  they  are  corrected  by  another 
crewmember. 


BASIC  QUALITY  11.  Supporting  information  and  actions  offered 

by  crew  (Info  Offered) 


Explanation: 

This  is  a  rating  of  the  extent  to  which  crewmembers  anticipate 
and  offer  support  information  and  support  actions  to  the  decision 
maker,  usually  the  pilot-in-command,  when  it  becomes  apparent 
that  a  decision  must  be  made  or  an  action  taken. 


Exampl es : 

•UH-60  Task  2016,  Perform  external  load  operations:  All 
crewmembers  will  assist  in  clearing  the  aircraft  and  will  provide 
adequate  warning  of  obstacles,  unusual  drift,  or  altitude 
changes . 

•UH-60  and  AH-64  Task  1081,  Perform  nonprecision  approach:  P  will 
call  out  the  approach  procedure  to  the  P* . 

Superior  Rating  (7) 

The  crew  recognizes  that  a  decision  must  be  made  and  offers 
suggestions  and  information  to  the  pilot-in-command.  The  crew 
checks  for  responses  that  indicate  understanding.  The 
information  is  repeated,  as  necessary,  to  ensure  that  the  pilot- 
in-command  understands  the  input.  Pilot-in-command  responses  can 
be  verbal  or  non-verbal  actions.  The  crew  seeks  information  and 
provides  it  to  support  decisions  and  actions.  The  crew 
frequently  offers  task  execution  support.  The  support  offered 
always  reflects  the  pilot-in-command's  task  needs.  Crews  are 
quick  to  offer  support  during  particularly  difficult  tasks  such 
as  obstacle  clearing. 


B-17 


Acceptable  Rating  (4) 


The  crew  recognizes  that!  a  decision  or  action  must  be  made  and 
offers  suggestions  and  information  to  the  pi lot -in-command.  The 
crew  sometimes  offers  task  execution  support.  Crewmembers 
usually  offer  obstacle  clearing  support. 


Very  Poor  Rating  (1) 

The  crew  does  not  offer  suggestions  and  inputs  to  support 
decision  making  or  actions.  Moreover,  it  often  appears  that  the 
crew  does  not  even  realize  that  a  decision  is  being  made.  The 
crew  generally  does  not  offer  its  services  to  support  task 
execution  for  other  crewmembers.  Crewmembers  may  fail  to  offer 
obstacle  clearing  support. 


BASIC  QUALITY  12.  Advocacy  and  assertion  practiced 

(Advoc /Assert ) 


Explanation: 

This  rating  evaluates  the  extent  to  which  crewmembers  advocate  a 
course  of  action  they  consider  best,  even  when  it  may  differ  with 
the  one  being  followed  or  proposed.  Note;  Except  under  extreme 
emergency  conditions  where  time  is  absolutely 

critical,  it  is  usually  in  the  crew's  best  interest  to  hear  the 
full  range  of  viewpoints  available. 

Exampl es : 

•UH-60  and  AH-64  Task  2083,  Negotiate  wire  obstacles:  Crew  will 
discuss  the  characteristics  of  the  wires  ...  to  determine  the 
method  of  crossing. 

.AH-64  Task  2044,  Perform  actions  on  contact:  Crew  will  discuss 
options  for  developing  the  situation. 

Superior  Rating  (7) 

Crewmembers  state  to  the  rest  of  the  crew  a  course  of  action  that 
they  consider  best.  They  clearly  explain  their  reasons  for 
believing  this  to  be  the  best  course.  Other _ crewmembers  listen 
to  the  argument  before  presenting  any  criticism  or  proposing 
alternate  courses.  Discussions  focus  on  the  strengths  and 
weaknesses  of  the  proposed  course  of  action,  not  on  the 
personality  of  the  crewmember  who  proposed  the  action. 
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Crewmembers  call  the  crew's  attention  to  changes  in  the  situation 
and  provide  information  that  is  essential  to  the  proper  execution 
of  another  crewmember's  task.  Crewmembers  pursue  feedback  to 
ensure  that  their  views  are  heard  and  understood.  Other 
crewmembers  expect  such  open  comments  and  view  them  as  positive 
contributions  to  mission  performance. 


Acceptable  Rating  (4) 

Crewmembers  state  their  support  for  a  course  of  action  or  suggest 
improvements  to  other  proposed  actions.  Each  crewmember  makes  an 
effort  to  explain  his  position  and  convince  others  to  concur  with 
him  on  the  course  of  action  to  be  taken.  Other  crewmembers  may- 
interrupt  with  their  views  and  alternatives.  Crewmembers  usually 
speak  out  when  they  recognize  a  departure  from  the  mission  plan 
or  standard  procedures  or  when  they  have  a  piece  of  information 
that  is  important  to  another's  task  execution.  Crewmembers  seek 
assurances  that  presented  information  has  been  received.  Other 
crewmembers  view  such  comments  as  constructive  and  not  as  a 
challenge  to  authority. 


Very  Poor  Rating  (1) 

The  crew  almost  never  suggests  a  course  of  action.  Crewmembers 
attempting  to  propose  a  course  of  action  may  be  cut-off  before 
they  can  propose  the  action  or  explain  the  rationale  for  that 
action.  Crewmembers  proposing  courses  of  action  may  receive 
personal  attacks.  The  crew  raises  few,  if  any  concerns. 
Crewmembers  may  even  fail  to  intervene  when  risks  such  as 
obstacles  or  poor  visibility  arise. 


BASIC  QUALITY  13,  Crew-level  a£Zer-actlon  reviews  accomplished 

(AAR) 


Explanation: 

This  rating  evaluates  the  extent  to  which  the  crew  reviews  and 
critiques  its  decisions  and  actions  during  or  following  a  mission 
segment,  during  low  workload  periods,  or  during  the  post  flight 
debrief.  Evaluate  the  crew  on  their  discussion  of  strengths  and 
weaknesses  (for  example,  what  was  done  wrong,  what  might  be  done 
better,  how  improvements  can  be  made,  and  what  was  done  very 
well)  in  flight  skills  and  aircrew  coordination. 

Superior  Rating  (7) 

The  entire  crew  reviews  and  critiques  its  decisions  and  actions 
throughout  the  mission,  including  the  pre-mission  planning  and 
rehearsal  process .  Crewmembers  review  factors  considered  in 
making  their  decisions,  identify  additional  options  or  factors. 


B-19 


•7y»TCWrr«vv.-*iv 


including  ways  to  "buy  time, "  that  should  have  been  considered, 
and  discuss  different  methods  of  weighing  information  in  the 
decision  process.  All  discussions  focus  on  behaviors  and 
information  and  carefully  avoid  any  "finger-pointing"  tones.  The 
focus  is  clearly  on  education  and  understanding  to  improve 
individual  and  collective  performance. 


Acceptable  Rating  (4) 

Senior  crewmember (s)  review  and  critique  the  crew's  decisions  and 
actions  during  problematic  segments  of  the  mission.  They 
determine  the  major  mistakes  in  the  crew's  actions  or  decisions 
and  identify  remedial  actions  or  alternative  options  for  future 
missions.  Although  the  critiques  are  intended  to  educate  the 
crew  and  to  improve  their  performance  during  future  missions, 
they  may  include  some  accountability  for  unsatisfactory 
performance . 


Very  Poor  Rating  (1) 

The  crew  either  fails  to  review  and  critique  its  mission 
performance  or  if  a  critique  is  performed,  it  is  punitive  or 
accusatory.  That  is,  the  critique  is  conducted  primarily  to 
assign  blame  for  unsatisfactory  performance.  Little  effort  is 
made  to  identify  lessons  learned  or  to  suggest  constructive  ways 
to  improve  future  performance. 
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Appendix  C 
Grade  Slips 


MANEUVER/PROCEDURE  GRADE  SUP  FOR  AH-64  AVIATORS 
For  use  of  this  form,  see  Aircrew  Coordination  Exportable  Training  Package  and  TC  1-214 

P _  Date _ 

CPG _ 

Instructor  or  evaluator  will  sign  in  the  first  unused  block  of  each  area  trained  or  evaluated 


STAN  EVAUTRAINING  TASKS 


CRBW  MISaON  BRIEFING 


PLAN  VFR  FLIGHT 


DA  FORM  5701 -R03PC> 


PREFLIGHT  INSPECTION 


ENG  START.  RUN-UP,  T/0  CKS 


HOI^  POWER  CHECK 


NORMAL  TAKEOFF 


TRAFFIC  PATTERN  FLIGHT 


FUEL  MANAGEMENT  PROCEDURES 


DOPPLS?  WWIGATION  (FS^ 


BEFORE  LANDING  CHECK 


VMC  APPROACH 


CONFINED  AREA  OPERATIONS 


SLOPE  OPERATIONS 


TERRAIN  FtlGHT  T|W<E0fF  . . 


TElWAINaiGHT 


NOE  DECELERATION 


TERRAIN  FLIGHT  APPROACH 


STANDARD  AUTOROTATION 


SIM  SINGLE  ENG  FAILURE  ALT 


SIM  SINGLE  ENG  FAILURE  OGE 


SINGLE-ENGINE  LANDING 


ECU  LOCKOUT  OPERATIONS 


TERRAIN  FLIGHT  NAVIGATK^ 


EMERGENCY  PR0C33URES 


HMO  PROCaaURESATHIRP 


MASKING  AND  UNKMSKING 


MAJ  US/ALLlEDn‘HREAT  EQUIP 


ACFT  SURVI V  EQUIP  (BS) 


AFTER-LANDING  TASKS 


MARK  XII  IFF  SYSTEM 


TADS  OPERATIONAL  CHKS  (FS) 


TADS  BORESIGHTING  (FS) 


TADS  SENSOR  OPERATIONS  (FS) 


IHADSS  BORESIGHT 


IHADSS  VIDEO  ADJUSTMENTS 


STAN  EVALfTRAINING  TASKS 


IHADSS  OPERATIONS 


DATA  ENTRY  PROCEDURES  (FS) 


ACFT  POSITION  UPDATE  (FS) 


TARGET  STORE  (FS) 


FIRING  POSITION  OPS 


ENGAGE  TGT  WITH  HELLFIRE 


ENGAGE  TGT  \MTH  ARCS 


ENGAGE  TGT  WITH  AWS  :  ;  : 


WPNS  INITIALIZATION 


TARGET  HANDOVER 


IHADSS  TARGET  TRACKING 


ORAL  EVALUATION 


NIGHT/NVD/EVAUTRAINING  TASKS 


CREW  MISSION  BRIEFING 


NVS  OPERATIONAL  CHECKS 


GROUND  TAXI 


HOVER  PO\ACR  CHECK 


HOVERING  FLIGHT 


NORMAL  TAKEOFF 


ROLLING  TAKEOFF  (BS) 


TRAFFIC  PATTERN  FLIGHT 


FUEL  MANAGEMENT  PROCEDURES 


PILOTAGE  &  DEAD  RECKONING 


DOPPLER  NAVIGATION 


VMC  APPROACH 


CONFINED  AREA  OPERATIONS 


SLOPE  OPERATIONS 


TERRAIN  FLIGHT  TAKEOFF 


TERRAIN  FLIGHT 


NOE  DECELERATION 


TERRAIN  FLIGHT  APPROACH 


SIM  SINGLE-ENG  FAILURE  ALT 


SINGLE-ENGINE  LANDING 


AIRCREW  COORDINATION  BASIC  QUALITIES 


2. 

3. 

4. 

5. 

6. 

7. 

8. 

9. 

10. 

11. 

12. 

PLAN 

DECI¬ 

WORK 

UNEXP 

INFO 

srr 

COMM 

INFO 

CROSS 

INFO 

AOVOC/ 

RE¬ 

HEARSE 

SION 

TECH 

LOAD 

EVENTS 

XFER 

AWARE 

ACK 

SOUGHT 

MON¬ 

ITOR 

OF¬ 

FERED 

ASSERT 

AIRCREW  COORDINATION  TRAINING  GRADE  SLIP 


MANEUVER/PROCEDURE  GRADE  SUP  FOR  AH-64  AVIATORS 


NIGHT/NVD  EVAUTRAJNING  TASKS 


TERRAIN  FLIGHT  NAVIGATION 


EMERGENCY  PROCEDURES 


UNUSUAL  ATTITUDE  RECOVERY 


IIMC  PROCEDURESA/HIRP 


MASKING  &  UNMASKING  (BS) 


TADS  OPERATIONAL  CKS  (FS) 


TADS  BORESIGHT  (FS) 


TADS  SENSOR  OPERATIONS  (FS) 


IHADSS  BORESIGHT! NG 


(HADSS  VIDEO  ADJUSTMENTS 


IHADSS  OPERATIONS 


NVG  FAILURE  (FS) 


ORAL  EVALUATION 


INST  EVAL/TRAINING  TASKS 


CREW  MISSION  BRIEFING 


IFR  FLIGHT  PLANNING 


ENG  START.  RUNUP.  T/0  CKS 


FUEL  MANAGEMENT 


BEFORE-LANDING  CHECK 


SIM  SINGLE-ENG  FAILURE  ALT 


EMERGENCY  PROCEDURES 


RADIO  NAVIGATION 


HOLDING  PROCEDURES 


UNUSUAL  ATTITUDE  RECOVERY 


RADIO  COMMUNICATIONS  PROC 


NONPRECISJON  APPROACH 


PRECISION  APPROACH 


IIMC  PROCEDURESA/HIRP 


AFTER-LANDING  TASKS 


ORAL  EVALUATION 


TRNG/HISSION/AODITIONAL  TASKS 


DO  FORM  365-4 


SIM  MAX  PERF  T/O  (BS) 


DECELERATION/ACCEL  (BS) 


ROLL-ON  LANDING  (BS) 


TERRAIN  FLT  MISSION  PLAN 


HIGH-SPEED  FLIGHT 


SIM  SINGLE  ENG  FAILURE  IGE 


STABILATOR  MALF  PROCEDURE 


INSTRUMENT  TAKEOFF  (BS) 


TWO-WAY  RADIO  FAILURE 


PINNACLE  OR  RIDGELINE  OPS 


HIGH/LOW  G  FLIGHT 


EVASIVE  MANEUVERS  (BS) 


MULTI  AIRCRAFT  OPERATIONS 


CALL  FOR/ADJUST  IND  FIRE 


VIS  SIGNAL  TECHNIQUES  (FS) 


LASER  SPOT  TRACKER  OPS  (FS) 


FARP  PROCEDURES 


ACTIONS  ON  CONTACT 


ID  TARGETS  WITH  TADS  (FS) 


SELECT  APPROPRIATE  WPN  SYS 


TGT  TRACKING  \MTH  TADS  (FS) 


OPERATE  ONBD  RECORDER  (FS) 


DIVING  FLIGHT  (BS) 


TECHNIQUES  OF  MOVEMENT  (BS) 


NEGOTIATE  WIRE  OBSTACLES 


TACTICAL  COMMO  AND  ECCM 


TRANS  TACTICAL  REPORT  (FS) 


NOTES: 

*  REQUIRED  FOR  NVG  EVAL 
+  REQUIRED  FOR  NIGHT  EVAL 

ENTER  S+.  S,  S-.  OR  U  IN  GRADE  BLOCK.  IF  GRADE  IS  S- 
OR  U  DUE  TO  AIRCREW  COORDINATION  INCLUDE  BASIC 
QUALITY  NUMBER(S) 


AIRCREW  COORDINATION  BASIC  QUALITIES 


1.  2.  3. 

CREW  PLAN  OECL 

CLI-  RE-  SfON 

MATE  HEARSE  TECH 


4. 

5. 

6. 

7. 

8. 

0. 

10. 

11. 

WORK 

UNEXP 

INFO 

srr 

COMM 

INFO 

CROSS 

INFO 

LOAD 

EVENTS 

XFER 

AWARE 

ACK 

SOUGHT 

MON¬ 

OF¬ 

ITOR 

FERED 

12.  13. 

ADVOC/  AAR 
ASSERT 
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BATTLE-ROSTERED  CREW  EVALUATION/TRAINING  GRADE  SLIP 
For  use  of  this  form,  see  Aircraft  ATM;  the  proponent  agency  is  TRADOC 


BATTLE- 

ROSTERED 

CREW 

EXAMINEES/ 

TRAINEES 


NAME 

PC: 

RANK 

PI: 

NONRATED  CREW  MEMBERS 

DUTY  SYMBOL 

NAME 

RANK 

EVALUATOR/ 

INSTRUCTOR 

NAME 

RANK 

UNIT: 

CREW  DATA 

TOTAL  BATTLE-ROSTERED 

DATE  DESIGNATED  A  BATTLE- 

CREW  HOURS 

ROSl  ERED  CREW: 

PURPOSE:  EVALUATION/TRAINING 

TIME  TODAY: 

CUMULATIVE  TIME: 

TYPE  AIRCRAFT: 

CREW  TASK  1 

D/N/NVD 

CREW  TASK  6 

D/N/NVD 

CREW  TASK  2 

D/N/NVD 

CREW  TASK  7 

D/N/NVD 

CREW  TASK  3 

D/N/NVD 

CREW  TASK  8 

D/N/NVD 

CREW  TASK  4 

D/N/NVD 

CREW  TASK  9 

D/N/NVD 

CREW  TASK  5 

D/N/NVD 

CREW  TASK  10 

.  D/N/NVD 

i  DAY 

NIGHT 

WX 

SIMULATOR 

NVG  NVS 

EVALUATOR/INSTRUCTOR  RECOMMENDATIONS 


□  (ISSUE)  (VALIDATE)  CREW  QUALIFICATIONS 


a  (SUSPEND)  (REVOKE)  CREW  QUALIFICATIONS 


CZI  REQUIRES  ADDITIONAL  (FLIGHT)  (ACADEMIC)  (SIMULATION  DEVICE)  TRAINING 


O  SEE  BACK  FOR  COMMENTS 


I  HAVE  DEBRIEFED  THE  EXAMINEES/TRAINEES  AND  INFORMED  THEM  OF  THEIR  STATUS. 

EVALUATOR'S/INSTRUCTOR’S  SIGNATURE: _ 

WE  HAVE  BEEN  DEBRIEFED  BY  THE  EVALUATOR/INSTRUCTOR  AND  UNDERSTAND  OUR  CURRENT 
STATUS. 

PC'S  SIGNATURE: _ 

PI'S  SIGNATURE: _ _ 

NONRATED  CREW  MEMBER'S  SIGNATURES: _ 


OVERALL  GRADE  FOR  THIS  FLIGHT  IS: 


U  NA 


OATE:__ 


DA  FORM  7121-R.  MAR  92 


COMMENTS 
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Appendix  D 
Exit  Interview 


AH-64  Instructor-Evaluator  Exit  Interview 


I.  Course  of  Instruction 

1.  Was  the  number  of  students  in  the  class  edsout  the  right  size 
for  this  training? 

Instructor  course  should  be  10-12  students;  no  more  than  14 
students  maximum. 

Same  number  of  students  for  the  student  course.  Eighteen  in 
the  student  course  was  too  many. 

[Does  mixing  platform  types  in  the  same  class  affect  training?] 

No  problem,  as  long  as  the  instructor  remains  aware  of  who 
is  in  the  class  and  asks  for  relevant  input. 

2.  Has  adequate  time  (or  too  little/too  much  time)  been 
allocated  for  each  segment  of  the  course?  In  answering  this 
question,  consider  both  the  Instructor  Course  and  the 
Student  course. 

Instructor  Course  Introduction  was  too  long.  Too  much 
repetition  of  material. 

Need  only  a  brief  introduction  of  about  1-2  hours. 

Covering  the  MOI  first  is  not  a  logical  flow.  Put  the  MOI 
at  the  end  of  the  regular  course  for  the  IPs/UTs. 

Unit  IPs  don’t  need  to  go  through  the  course  twice.  They 
should  go  through  the  course  the  first  time  with  their 
battle-rostered  crewmember. 

Need  to  allow  the  IPs/UTs  to  "teach  back"  the  material  to 
the  trainers  as  a  check  on  their  ability  to  teach.  This 
could  be  a  block  where  different  IPs  take  a  specific  section 
to  teach. 

3.  How  many  simulator  sessions  [AH]  or  flight  periods  [OH]  are 
required  in  the  Instructor  Course?  In  the  Student  Course? 

Okay  as  is  with  four  missions  minimum  in  each  course. 

4.  What  effect,  if  any,  did  crew  coordination  trained  IPs  and 
UTs  operating  with  their  battle-rostered  crewmember  have  on 
the  training? 

Tends  to  degrade  the  other  crew  member's  training. 

IPs  have  a  hard  time  not  remembering  scenario  details  and 
artificially  reacting  to  them  because  they  have  seen  them 
before . 

IPs  tend  to  either  under  react  or  overreact  degrading 
training  for  both  crewmembers. 


Should  a  simulator  session  where  IP/UT  crews  rate  each  other 
be  used  for  practice  evaluations  in  the  Instructor  Course  or 
are  the  rating  exercises  using  video  segments  adequate? 

If  people  are  willing  to  allow  it,  you  could  use  the  initial 
evaluation  flights  as  a  practice  mission  tape  to  evaluate. 
Another  option  is  to  develop  a  "reenacted"  video  taped 
mission  with  known  crew  coordination  errors/events  for 
discussion. 

Taped  missions  can  be  stopped  to  allow  discussion,  whereas 
you  can't  do  this  as  easily  in  the  cockpit. 

More  people  can  watch  a  video. 

What  effect  [AH] ,  if  any,  did  the  pre-training  evaluation 
mission  in  the  simulator  have  on  the  classroom  instruction 
part  of  the  Instructor  Course  and  the  Student  Course? 

It  was  a  good  basis  to  start  from  and  enhanced  thoughtful 
discussion  in  both  classes. 

It  makes  it  easier  to  tie  all  the  segments  together  and 
understand  what  is  expected. 

Instructors  were  able  to  refer  back  to  these  initial 
evaluation  missions  often  in  the  academic  instruction. 

Did  you  read  the  read-ahead  package  materials?  If,  yes,  did 
the  read-ahead  packages  reduce  the  amount  of  time  spent  on 
specific  subjects?  Did  they  enhance  the  flow  of  the  course? 
Did  you  review  the  homework  assignments  at  the  beginning  of 
each  day's  instruction? 

Most  of  the  IPs  did  not  read  the  material,  except  for  the 
two-page  pretraining  evaluation  item.  This  material  was 
given  over  a  weekend  that  was  their  first  weekend  off  in  a 
month.  There  was  not  much  motivation  to  spend  the  weekend 
reading  this  material. 

The  length  of  the  course  leaves  you  bored,  even  though  the 
material  is  good.  Read-aheads  add  to  boredom. 

Material  should  be  retained  and  used  as  a  continuation 
training  topic  on  safety  stand  down  days. 

The  accident  briefs  were  interesting  and  many  crew  members 
read  them. 

Need  more  videos  to  support  the  case  studies — they  were 
interesting  and  the  examples  do  not  have  to  be  specific  to 
the  platform  that  the  student  flys. 

Practice  AARs  often  had  discussions  diverging  off  talking 
about  emergency  procedures  rather  than  the  crew  coordination 
aspects  of  the  incident. 
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Did  the  Instructor  Course  adequately  prepare  you  to  teach 
the  Student  course? 

Academics-wise,  no.  There  was  still  confusion  on  the 
specific  points  to  teach  in  each  section.  This  needs  better 
coverage  in  an  MOI  section  that  stresses  certain  points  to 
teach  (better  covered  at  the  end  of  the  course) . 

Some  of  the  IPs  had  never  had  training  on  how  to  teach  from 
the  platform,  how  to  ask  questions,  how  to  draw  out 
discussion,  etc.  Need  to  add  platform  instruction 
techniques  to  the  MOI  section.  CAE-Link  course  has  a 
platform  instruction  technique  section  in  its  MOI  course. 

A  little  time  in  the  course  to  practice  teaching  the 
material  would  provide  a  "confidence  builder"  for  the  IPs. 
The  time  allotted  in  the  course  was  adequate;  but,  they 
didn't  know  what  to  do  with  the  time  in  terms  of  practicing. 
The  initial  cadre  might  want  to  consider  a  follow-up  visit 
to  check  on  the  IPs'  ability  to  teach.  This  wasn't  a 
problem  in  the  unit  because  they  were  fresh  out  of  the 
course.  If  you  wait  a  month,  you  may  have  more  problems 
that  require  a  refresher  "teach  back"  event. 

Need  someone  who  has  taught  the  course  before  to  monitor 
new  instructors.  This  would  be  a  big  help. 

Are  there  any  Instructor  Course  segments  (for  exanple,  MOI, 
evaluation,  scenario  development)  that  should  receive  more 
or  less  enphasis? 

See  item  8  comments  on  MOI  above. 

Need  more  videos  to  support  academic  instruction. 

Scenarios 

Were  the  evaluation  scenarios  of  about  the  correct  level  of 
difficulty? 

Yes.  The  basic  scenarios  were  a  good  starting  point.  The 
units  were  not  using  the  simulator  to  this  extent  before. 

Use  of  standardized  scenarios  would  help. 

Avoid  making  the  scenarios  so  difficult  that  they  take  all 
of  the  crews'  concentration  away  from  learning  the  crew 
coordination  techniques. 

Were  the  evaluation  scenarios  reasonably  realistic? 

Yes. 

Developing  a  script  was  helpful  since  unit  instructors  had 
never  done  that  before  for  their  unit  scenarios. 

Need  some  variety — e.g.,  movement  to  contact  scenario,  deep 
attack. 

Use  of  the  scout  aircraft  was  a  little  overdone,  as  compared 
to  what  the  unit  really  does — e.g.,  more  use  of  the  other 
gun  aircraft  rather  than  scout  aircraft. 
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k. 


Incorporate  unit  specific  METL  tasks  where  possible. 

3.  Was  there  enough  pre-mission  planning  time  for  the  crews? 

Yes,  always. 

4.  Did  the  scenarios  allow  adequate  demonstration  and 
observation  of  the  13  crew  coordination  Basic  Qualities? 

Yes,  they  were  adequate. 

Some  of  the  emergency  malfunctions  could  be  more  complex, 
requiring  more  than  one  step  to  be  taken  by  the  crew. 
Simulator  limitations  restrict  the  complexity  of  the 
malfunctions.  Need  to  look  for  malfunctions  that  require 
crew  interaction — IMC  was  a  good  training  situation. 

Some  BQs  were  not  used  very  often — e.g.,  some  of  the 
information  transfer  BQs  were  lumped  together. 

5.  Did  the  crew-level  AAR  checklist  adequately  cover  all 
aspects  of  the  mission?  Should  any  items  be  added  or 
deleted? 

A  lot  of  people  used  it,  but  didn't  follow  it  verbatim — 
tailor  to  specific  mission. 

The  list  provides  good  reminders  of  what  you  should  cover. 
Reviews  were  conducted  mainly  by  mission  segment. 

A  copy  should  be  posted  in  the  after  action  room. 

Ill .  Evaluation 

1.  Were  mission  videotapes/audiotapes  of  pre-mission  planning, 
flight,  and  crew-level  after  action  review  segments  helpful 
to  instruct  and  evaluate?  If  yes,  how  were  they  helpful? 

The  flight  video  tapes  were  especially  helpful  to  review 
crew  and  individual  skills.  The  tapes  of  the  premission 
brief  and  AAR  were  not  used. 

Tapes  were  a  good  tool  for  answering  questions  and  resolving 
disputes  over  mission  events  and  crew  behavior.  Used 
simulator  page  printouts  also. 

Found  the  tapes  useful  to  critique  the  CPG's  scan  patterns. 

2.  Are  audio  recordings  [OH]  and  evaluator  observations  of 
flight  segments  adequate  to  instruct  and  evaluate  crew 
coordination  skills? 

Video  tapes  are  essential. 
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During  your  instructor  debriefing,  did  you  review  the  whole 
videotape/audiotape  or  did  you  refer  only  to  specific 
segments? 

IPs  didn't  look  at  the  entire  tape,  but  found  it  useful  to 
review  segments  containing  critical  events . 

It  was  difficult  to  find  the  exact  tape  segments  to  return 
to.  Consider  recording  in  EP  mode  to  allow  quicker  "fast- 
forward"  scanning  of  tape. 

What  general  comments  did  the  aircrews  make  as  they 
observed/listened  to  their  tapes? 

Crews  often  catch  mistakes  themselves  by  looking  at  the 
video  tapes. 

Tapes  allow  you  to  step  outside  yourself  and  look  at  the 
mission  objectively.  Draws  out  what  the  crew  was  thinking 
at  the  time. 

Crews  should  be  required  to  watch  their  pre-  and  post¬ 
training  evaluation  tapes  in  their  entirety,  even  though 
this  requires  a  lot  of  time. 

If  video  recording  of  flight  segments  is  not  possible,  can 
objective  and  reliable  crew  coordination  evaluations  be 
conducted  in  the  aircraft?  For  exanple,  can  evaluations  be 
conducted  from — 

a.  A  non-flying  station  (back  seat  or  jun^  seat)  [OH]? 

Not  applicable. 

b.  A  flying  station  as  a  crewmember  [AH  &  OH]? 

Not  really  as  good  as  you  can  evaluate  as  a  third  person  in 
the  simulator. 

Use  only  as  a  last  resort. 

c.  Another  aircraft  [OH]? 

It  could  be  done,  but  only  50%  effective  due  to  lack  of 
intracrew  observations. 

Could  use  mission  tapes. 

Were  the  behavioral  anchors  useful  or  not  useful  to  you  in 
achieving  objective  and  reliable  ratings  of  crew 
performance?  How  did  you  use  the  behavioral  anchors? 

They  were  very  useful,  particularly  for  new  evaluators. 
Evaluations  are  still  subjective. 
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7. 
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Did  the  video  segments  [used  in  the  Instructor  Course 
evaluation  workshop  and  practice  evaluations]  provide 
adequate  opportunity  for  practicing  your  application  of  the 
rating  scales? 

Yes.  See  section  I,  item  5. 

8 .  Here  you  reluctant  to  give  crews  task  and  mission  grades 
below  "satisfactory"  or  crew  coordination  ratings  below 
"acceptedsle"?  If  yes,  why? 

No.  Behavioral  anchors  were  useful  in  justifying  bad 
grades . 

Showing  evidence  on  tape  was  useful  in  justifying  bad 
grades . 

Most  of  the  time  the  crews  knew  when  they  made  mistakes . 

You  don’t  want  too  many  unsatisfactory  ratings;  otherwise, 
you  kill  motivation  to  improve. 

9.  How  often  did  you  refer  to  the  written  descriptions  in  the 
behavioral  anchors? 

Occasional  reference  at  first,  then  not  necessary. 
Infrequent  review  of  superior  anchors  as  most  of  the  time 
only  the  "acceptable"  rating  was  discussed  in  class. 

10.  Was  the  satisfactory  plus  (S+) ,  satisfactory  (S) ,  and 
satisfactosry  minus  (S-)  grading  system  helpful? 

Yes.  Very  helpful. 

The  use  of  S(-)  was  good  as  a  "wake  up  call"  that  could  be 
used  to  motivate  without  killing  the  person's  enthusiasm. 
Could  provide  some  S(+)  marks  to  balance  things  out  and  let 
the  crews  know  that  the  evaluation  isn't  all  negative. 

IV.  General  Observations 

1 .  What  is  your  overall  inqpression  of  the  adequacy  of  the 

aircrew  coordination  training  provided?  Do  you  have  any 
recommendations  for  in5>rovement? 

Overall  the  course  was  great. 

Academics  could  be  shortened  as  mentioned  before,  for 
example,  place  the  MOI  segments  at  the  end  of  the  course. 
Don't  change  the  simulator  training. 

Produce  more  AH- 64  related  video  segments. 

Observed  major  improvement  in  some  crews  in  terms  of 
workload  management  and  assertiveness. 

Good  crews  improved  some.  New  crews  made  great 
improvements.  Even  crews  with  many  hours  together  improved 
as  a  crew. 

All  crews  learned  from  the  training. 
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What  is  your  overall  inqpression  of  the  adequacy  of  the 
evaluation  training-  provided?  Do  you  have  any 
recommendations  for  in^rovement? 

Consider  reenacting  flight  segments  for  video  tape  and  use 
in  class  as  practice  evaluations. 

What  is  your  overall  in^ression  of  the  adequacy  of  the 
aircrew  coordination  evaluations?  Do  you  have  any 
recommendations  for  in^rovement? 

They  were  fair  and  objective. 

The  number  of  BQs  allowed  a  fair  evaluation  and  good 
feedback. 

Instructors  need  to  consider  the  mission  circumstances  in 
making  ratings  to  ATM  standards. 

Should  use  the  same  IP  to  conduct  the  pre-training 
evaluation  and  last  evaluation  flight,  with  another  IP  doing 
the  training  in  between. 

Did  anything  presented  in  the  classroom  or  hands-on 
instruction  suggest  actions  that  could  potentially 
conqpromise  flight  safety?  If  yes,  please  provide  specific 
examples . 

No.  Unfortunately,  the  CMS  does  not  allow  you  to  exercise 
the  2-challenge  rule  because  it  can't  transfer  controls 
rapidly  without  10  intervention. 

Do  you  have  any  questions,  concerns,  or  recommendations  that 
you  would  like  to  ask  or  convey  to  the  crew  coordination 
project  staff? 

Much  of  the  course  contains  material  that  has  been  discussed 
before  within  the  unit;  however,  the  course  effectively 
brings  it  all  together  and  writes  it  down  in  one  place. 

The  2-challenge  rule  needs  to  be  clearly  defined  and 
discussed  in  depth. 
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